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Message  to  the  Europe  Chapter 


Harold  van  Cott,  Ph.D.,  President 
Human  Factors  and  Ergonomics  Society 


Professor  Cavonius,  Ladies  and  Gentlemen:  Greetings  to  all  of  you  here  in  Bochum  at  this 
meeting  of  the  Europe  Chapter  of  the  Human  Factors  and  Ergonomics  Society.  I  had  hoped  to 
be  here  with  you  today,  but  an  unavoidable,  personal  scheduling  conflict  prevented  it.  None- 
the-less,  on  behalf  of  the  Society’s  Executive  Council,  I  would  like  to  wish  you  well  for  a 
successful  conference. 

In  September,  I  attended  the  1997  41"'  Annual  Meeting  of  the  HFES  in  Albuquerque,  a  small, 
historic  city  in  central  New  Mexico.  It  was  a  busy  week  of  renewed  friendships,  symposia, 
papers,  posters,  receptions,  banquets,  and  tours.  Late  September  is  a  good  time  to  be  in 
Albuquerque.  Seeing  the  awesome  Sandia  mountain  range  emerge  from  morning  twilight  into 
a  pure  blue  sky  invigorates  the  soul  for  the  day  ahead.  The  days  were  long  but  gratifying. 
Evening  meals  with  green  chili  stew,  ostrich  and  buffalo  steak  were  a  novelty  to  tastes  formed 
in  the  Eastern  U.S, 

In  1956,  the  HFES  had  its  first  meeting  in  Tulsa,  Oklahoma.  About  150  charter  members,  who 
had  organized  what  was  then  called  the  Human  Factors  Society  of  America,  were  there.  I  was 
among  them:  green  but  enthusiastic.  This  year  over  1.200  members  from  aU  over  the  world 
attended  the  Albuquerque  meeting.  I  was  there  and  became  the  41"*  President  of  the  Society. 
Today,  I’m  gray  but  still  roaring  to  go.  This  is  an  exiting  time  to  be  a  member  of  HFES.  It  will 
become  more  exiting  as  the  society’s  strategic  plan  continues  to  unfold.  As  members  assume 
society  leadership  roles  and  take  on  program  activities,  the  accomplishments  of  the  long-range 
plan  will  be  increasingly  felt,  the  society  will  flourish,  and  our  profession  will  grow.  This 
strong  European  Chapter  is  but  one  proof  of  the  growing  global  human  factors  and 
ergonomics  community.  We  are  delighted  that  distinguished  colleagues  such  as  you  are 
members. 

Today  I  would  like  to  tell  you  about  the  accomplishments  in  1997  and  the  plans  for  1998  of 
the  Human  Factors  and  Ergonomics  Society, 

A  lot  was  accomplished  in  1996/7.  Our  Web  Site  was  running  smoothly  and  was  used 
extensively.  The  directory  of  graduate  programs  in  human  factors  education  and  the  author’s 
guide  to  preparing  manuscripts  for  publication  were  put  on  the  Web  Site. 

Informing  others  outside  the  field  of  the  benefits  of  applying  human  factors  data  and 
principles  has  always  been  a  problem.  Last  year,  an  illustrated  report.  Good  Ergonomics  is 
good  Economics,  was  published  as  a  removable  insert  in  our  magazine.  Engineering  in 
Design.  These  case  histories  drawn  from  around  the  world,  on  the  costs  and  benefits  of  human 
factors  and  ergonomics  interventions,  are  available  in  either  paper  or  electronic  format  from 
the  Society’s  email  address  thfes@compiiserve.com) .  Slides  and  text  of  these  stories  may  also 
be  obtained  from  the  Central  Office  for  use  in  giving  presentations.  The  success  stories  will 
be  expanded  and  updated  as  members  augment  the  current  file  of  stories  and  pictures  with 
material  form  their  own  experiences.  I  invite  you  to  do  so. 

Feedback  from  focus  groups  and  from  a  survey  of  all  members  showed  that  HFES  should  put 
more  effort  into  reaching  out  to  government  and  industry  with  education  and  information.  In 
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the  past  year,  and  with  the  combined  strength  derived  from  teaming  with  the  Federation  of 
Behavioral  and  Cognitive  Sciences  and  the  American  Psychological  Association,  the  HFES 
participated  in  programs  to  educate  staff  of  the  U.S.  Congress  on  the  value  of  the  discipline 
and  the  importance  of  supporting  funding  for  research  and  applications.  Presentations  and  an 
exhibit  were  made  at  the  Medical  Device  and  Manufacturing  Show.  A  policy  and  guidelines 
for  conducting  advocacy  and  outreach  efforts  was  being  developed. 

Given  that  more  of  our  members  are  moving  into  small  business  and  independent  consulting, 
we  decided  to  give  workshops  on  forming  and  managing  efforts  in  these  areas  for  the  first 
time  in  Albuquerque.  Enrollment  and  participant  enthusiasm  was  high,  and  snnilar  workshops 
are  contemplated  for  1998. 

With  funding  in  hand  from  two  founding  partner  companies-  Sun  Microsystems,  Inc.  And  the 
Eastman  Kodak  Company-  the  HFES  Institute  became  a  dream  turned  in  reality.  The  Institute 
will  provide  a  badly  needed  mechanism  for  procuring  research  data  needed  to  fill  critical  data 
gaps  in  new  industry  standards,  guidelines  and  best  practices. 

Volunteers  are  the  life-blood  of  HFES.  More  are  needed  if  we  are  to  enhance,  strengthen  and 
extend  our  efforts.  But  good  training  is  required  for  volunteers  to  become  truly  effective.  This 
year,  again  for  the  first  time,  a  Leadership  Day  for  local  and  student  chapter  presidents  and 
technical  group  chairs  was  held.  Over  80  enthusiastic  participants  shared  experiences, 
identified  common  problems,  and  started  to  build  plans  for  solutions  to  them.  Many  of  those 
present  had  not  met  before  and  formed  ties.  It  was  a  good  start  on  the  way  to  our  goal  of 
amplifying  the  capability  of  the  Executive  Council  to  plan  and  manage  our  many  new 
initiatives. 

Here  are  some  examples  of  new  efforts  scheduled  for  1997-98: 

Two  new  collections  of  papers  from  the  Proceedings  will  be  published  on  topics  of  high 
current  professional  and  public  interest.  One  will  be  on  design  for  the  aging  population.  The 
other  will  cover  papers  on  musculosceletal  disorders  and  manual  handling. 

Two  important  surveys  are  planned  for  1998.  One  will  be  a  survey  of  member  needs  and 
preferences  for  the  electronic  delivery  of  mail.  The  other  wiU  be  a  detailed  survey  of  salaries. 

If  your  arms  ached  or  your  wallet  shrunk  from  canying  or  mailing  home  the  two  volumes  of 
annual  meeting  proceedings,  be  of  good  cheer.  Relief  form  transporting  the  growing  bulk  of 
the  Proceedings  is  in  sight  Next  year,  they  will  be  available  on  CD-ROM  in  a  searchable 
format,  as  well  as  in  traditional  hard  copy.  In  addition,  a  portion  of  cumulative  index  of  our 
journal  may  be  included  on  a  trial  basis  on  CD-ROM. 

Students  are  always  broke.  Next  year,  student  members  will  be  able  to  receive  a  50%  discount 
off  full  member  dues.  For  the  first  time  two  years  following  graduation  while  they  are  getting 
professionally  established. 

Next  year  the  Executive  Council  will  continue  strategic  planning,  monitor  current  programs, 
plan  new  initiatives,  and  invite  feedback  and  ideas  from  all  members.  To  do  all  of  these  things 
well,  the  Council  will  meet  quarterly.  Additional  outreach  efforts,  leadership  forums, 
practitioner  workshops  and  issues  of  membership  recruitment  and  retention  will  be  given  high 
priority  at  these  meetings. 

As  for  my  own  agenda  as  President,  I  plan  to  revive  the  dormant  Technical  Advisory  Group  to 
advise  the  Council  on  emerging  opportunities  for  the  application  of  our  science  and 
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technology,  and  for  the  actions  that  HFES  can  take  to  be  proactive  in  the  pursuit  of  these 
opportunities.  After  learning  about  the  struggles  of  our  chapters  simply  to  survive,  to  hold 
meetings  and  to  communicate  with  members,  I  plan  to  explore  ways  to  help  chapters  and  the 
technical  groups  to  establish  listservers  to  enhance  communication  among  tiieir  members  and 
with  other  chapters  and  HFES.  I  also  plan  to  visit  local  and  student  chapters.  I  will  continue  to 
support  HFES’  standards  activities  and  the  new  HFES  Institute,  and  I  will  work  with  the 
Council  of  Technical  Groups  on  a  plan  to  create  a  “NewsletterDigest”,  a  publication  drawing 
from  the  best,  most  interesting  and  useful  articles  in  the  newsletters  published  by  the 
Technical  Groups. 

In  concluding,  I  invite  you,  the  members  of  the  Europe  Chapter,  to  send  me  suggestions  and 
ideas  about  how  to  strengthen  the  Society  and  advance  the  science  and  practice  of  human 
factors  and  ergonomics.  Your  perspectives  as  colleagues  who  live  and  work  outside  of  North 
America,  wiU  provide  us  with  insights  on  new  research  methods  and  applications  that  will 
enrich  our  conunon  endeavors.  My  email  address  is  HPV ANCQTT @  AOL.COM- 

Thank  you  for  inviting  me  to  attend  this  meeting.  I  am  sorry  that  I  can  not  be  with  you.  I  trust 
that  you  will  find  his  to  be  a  worth  while  conference.  Have  fun  as  well! 
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Introduction 

The  speed  of  technical  development  in  the  area  of  new  information  and  communication 
technologies  can  hardly  be  correctly  estimated.  For  trend  researchers,  politicians,  and 
“economic  gurus“  Multimedia  when  associated  with  terms  such  as  information  society, 
data  highway  and  digital  era  promises  solutions  for  the  greatest  problems  of  the  2F* 
century:  worldwide  peace,  global  growth  and  unemployment.  Technical  opportunities 
that  are  concealed  behind  the  shimmering  term,  Multimedia,  are  definitely  impressive. 
The  combination  of  communication  systems,  telephone,  data  networks,  and  communica¬ 
tion  satellites  with  other  means  of  communication  such  as  television,  computer,  fax  ma¬ 
chine,  and  radio  offers  users  the  perspective  of  worldwide  interactive  connection.  In  ad¬ 
dition  to  real  corporations,  more  and  more  virtual  firms  will  be  developed,  creating  a  so 
called  “cyber  industry“.  These  virtual  corporations  are  agile,  global,  “cybemetic“,  and 
fast  learning  companies,  which  will  expedite  the  evolution  of  the  business  worlds  in  a 
decisive  manner.  Cyber  corporations  are  part  of  modem  economic  systems,  which  show 
similar  behavioral  patterns  to  biological  ecosystems.  They  connect  multiple  industries 
and  compete  for  synergetic  effects  and  predominance  in  the  jungle  of  competition.  James 
Martin  distinguishes  three  kinds  of  intelligent  Darwinist-like  evolution  (Martin,  1997a). 
The  primary  type  of  evolution  relates  to  the  modification  of  a  product  or  service  while 
maintaining  a  predefined  manufacturing  process  and  a  fixed  structure  of  the  corporation. 
The  secondary  type  of  evolution  refers  to  the  modification  of  the  production  process,  the 
methodology  or  the  workflow.  In  this  context  the  structure  of  the  corporation  can  be 
adapted  appropriately.  The  tertiaiy  type  of  evolution  investigates  factors  outside  of  the 
company,  termed  the  framing  condition,  i.e.  relations  to  other  companies. 

My  long  standing  experiences  as  an  entrepreneur  proved  these  evolutions  not  only  to  be 
typical  for  cyber  enterprises  but  also  necessary  for  the  development  of  corporations  to¬ 
day.  Yet,  the  concentration  on  the  tertiary  type  of  evolution  will  be  considerably  in¬ 
creased  as  more  cyber  corporations  emerge.  Multimedia  is  literally  the  combined  use  of 
different  media,  more  precisely,  use  of  such  media  that  utilizes  different  human  senses 
such  as  the  visual  and  the  auditory  systems.  So  far.  Multimedia  has  had  several  different 
effects.  Multimedia  is  not  only  comprised  of  multiple  modes  of  information,  but  also 
computer  integration  and/or  computer  support.  The  overall  effect  is  a  computer  sup¬ 
ported  media  integration  process.  The  user  has  time-independent  access  on  information, 
digital  processing,  storage  and  proliferation  of  this  data  and  also  the  option  to  interact 
and  communicate  at  any  time  (Hermanns;  Suckrow,  1993.  Grabner;  Lang  1992.  Stippel, 
1992). 

For  my  subsequent  explanations  and  arguments,  I  would  like  to  use  the  term  Multimedia 
synonymously  with  Multimedia  techniques  for  new  information  and  communication 
technologies.  I  choose  to  do  this  because  there  are  a  great  number  of  different  interpreta¬ 
tions  and  definitions  for  Multimedia,  even  if  these  compose  a  more  complex  scope  than 


original  sense.  For  reflections  on  changes  in  business  and  management  this  definition  is 
very  useful.  I  do  not  want  to  concern  myself  with  all  technical  parameters  at  this  time. 

Many  authors  have  dealt  with  Multimedia  definitions  under  the  viewpoint  of  assessing 
social  aspects  of  new  technologies.  Many  more  authors  have  described  social  aspects  of 
new  technology  without  even  mentioning  Multimedia.  The  reason  for  is  that  Multimedia 
is  a  comparatively  new  term,  though  selected  elements  of  it  have  long  existed,  subsumed 
under  terms  like  new  techniques,  computer  networking,  hypermedia  etc.  Many  books 
included  in  the  bibliography  therefore  do  not  explicitly  address  Multimedia  but  never¬ 
theless  contribute  significantly  to  social  aspects  of  computer  applications,  the  influence 
of  computer  networking  to  management  and  the  consequences  of  connecting  enterprises 
worldwide. 

Unfortunately  I  cannot  cite  and  evaluate  the  literature  covering  these  topics  in  detail.  Yet 
a  number  of  books  or  studies  will  be  mentioned  and  cited  in  brief  throughout  this  paper 
if  necessary  to  further  clarify  my  statements. 

Multimedia  in  the  Sociological  Evolution 

The  protagonists  of  new  technology  address  Multimedia  as  the  most  important  revolution 
since  the  invention  of  fire  (Spiegel  Spezial,  1996a).  The  term  Information  Society  is  fre¬ 
quently  used  here  without  analyzing  the  true  content  of  the  term.  The  following  basic 
understanding  which  is  of  crucial  importance  is  frequently  forgotten:  “Society  cannot  be 
reduced  to  information.  Economical,  ecological,  social,  and  psychological  conditions 
form  the  foundation  of  life  and  not  only  information  (Hdfling,  1996). 

There  are  currently  four  extreme  views  of  Multimedia.  They  are: 

•  A  warning  that  Multimedia  destabilizes  established  sound  stmctures  in  all  areas, 

•  The  tendency  to  expect  a  great  “hausse”  (boom  or  bull-market)  for  society,  econom¬ 
ics,  and  consumer,  a  true  thrill, 

•  The  expectation  of  a  “baisse”  (fall  or  bear-market),  a  doomsday-like  situation, 

•  The  belief  that  the  world  will  not  change  despite  technical  innovations. 

Probably  none  of  the  four  statements  can  truly  characterize  the  consequences  of  this  new 
technology  alone.  During  a  study  initiated  by  the  Office  of  Research  on  Consequences  of 
Technology  from  the  German  Parliament  in  Bonn,  different  application  areas  of  Multi- 
media  technologies  were  covered,  including  the  following: 

•  Multimedia  in  Business  Applications 

•  New  Media  worlds  in  private  households 

•  Multimedia  in  the  public  domain,  i.e.  for  citizens  and  community 

•  Learning  by  Multimedia 

•  The  new  Multimedia  vocabulary 

•  Computer  support  for  radio 

The  study  comprises  nearly  ten  pages  of  consequences  and  suggestions  (Riem;  Wingert, 
1996),  that  make  clear  that  Multimedia  greatly  influences  all  areas  of  life  and  will  result 
in  profound  changes  in  conmiercial  activity.  I  do  not  want  to  restrict  my  analysis  to  the 
business  and  management  processes  as  companies  are  embedded  in  sociological  framing 
conditions,  especially  since  the  tertiary  evolution  constitutes  the  greatest  challenges. 
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Changes  in  the  social  attitudes  of  people  with  respect  to  behavior  while  buying,  relaxing 
or  vacationing  have  a  direct  economic  effect  on  values  in  the  ecological  situation.  This 
is  a  very  important  issue  for  me.  I  am  interested  to  see  how  people  will  take  the  chal¬ 
lenge  of  Multimedia,  how  they  will  use  their  chances,  see  its  risks  and  understand  the 
innovative  development  of  the  information-  and  communication  technologies  that  wiU 
become  part  of  new  organizational  structures  in  business.  I  am  eager  to  see  new  concepts 
of  marketing  and  sales  in  certain  aspects  of  principal  policies.  Learning  will  develop  to 
be  a  complex  challenge.  From  my  understanding  this  challenge  will  not  only  relate  to 
“learning  with  Multimedia“,  but  to  learning  for  and  because  of  Multimedia.  It  cannot  be 
overseen  that  in  the  public  and  in  management  on  many  different  levels  there  is  already  a 
certain  level  of  knowledge  regarding  the  existence  of  Multimedia  applications.  Society 
on  the  other  hand  possesses  only  partial  competence  to  use  these  new  opportunities. 
Moreover,  the  risks  of  multimedia  technologies  are  perceived  very  differently  in  practice 
work,  everyday  use,  and  by  small  and  medium  sized  companies  even  though  the  theoreti¬ 
cal  aspects  are  well  defined.  There  is  stiU  very  little  routine  skill,  know-how,  and  user 
experience  available  in  these  technologies  on  a  broad  scale.  It  is  not  only  a  requirement 
of  school  education  to  further  develop  these  skills,  but  it  is  also  an  essential  task  for  the 
management  in  businesses.  This  point  I  will  further  elaborate  on  later  in  this  paper. 
Moreover,  we  will  need  a  realistic  calculation  of  national  economy  and  business  admini¬ 
stration  related  facts  to  assess  the  ever  accelerating  physical  and  moral  deterioration  of 
new  information-  and  communication  technologies  and  software. 

Only  in  the  evolutionary  context  of  society,  a  clearer  understanding  of  Multimedia  can  be 
developed  which  is  neither  euphoric  nor  apocalyptic,  but  realistic.  Businesses  will  be 
forced  to  approach  these  new  technologies  with  a  much  stronger  emphasis.  They  should 
do  this  on  the  basis  of  their  critical  engagement. 

New  Challenges  to  Democracy  and  Participatory  Leadership  in  Industry 
General  access  to  information  has  a  basic  and  profound  sociological  importance  and  is 
an  important  prerequisite  for  a  democratic  society  and  participatory  leadership  on  the 
business  level. 

In  conjunction  with  the  possibilities  to  spread  out  new  multimedia  techniques,  numerous 
new  formats  to  democratize,  monopolize,  globalize,  and  respectively  regionalize  society 
and  management  will  result.  The  jurisdictional  framework  for  the  new  information  and 
communication  services  has  not  yet  been  defined.  Germany  is  one  the  first  countries  in 
the  world  to  have  developed  such  a  framework.  These  new  established  standards  focus 
on  the  principle  of  extensive  deregulation,  which  promises  fi*ee  access,  responsibility, 
and  data  protection  specific  to  different  areas.  Yet,  present  laws  are  not  fully  accepted  by 
the  German  economy.  The  main  criticism  exist  against  the  following  split  governmental 
responsibility  for  different  electronic  services: 

•  German  federal  states  are  responsible  for  TV-shopping  and  Videotext, 

•  Internet  and  telebanking  are  under  the  sovereignty  of  the  Federal  Government, 

There  are  no  regulations  within  corporations  regarding  management. 

The  Federal  Association  of  German  Industry  expects  an  increase  in  uncertainty  with  re¬ 
spect  to  legal  problems  and  investment  policies.  They  are  also  afi*aid  that  many  busi- 
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nesses  will  leave  the  country,  because  increased  requirements  of  data  protection  will 
highly  influence  the  flow  of  data  within  the  corporation.  The  further  development  of  the 
legal  framework  is  of  crucial  importance  to  the  full  use  of  labor  potentials  as  documented 
by  a  study  of  the  Council  of  Economic  Advisors  to  the  United  States  of  America.  Ac¬ 
cording  to  the  study,  a  significant  increase  in  return  from  investment  could  be  achieved 
by  reduction  of  legal  uncertainties,  the  elimination  of  restrictive  regulations  and  the  pro¬ 
motion  of  competition  in  telecommunication  and  in  the  telecommunication  industry. 
Besides  legal  regulations,  restrictions  of  access  to  use  new  technologies  with  respect  to 
costs  have  to  be  taken  into  account.  The  new  technologies  presumably  guarantee  their 
users  an  equal  access  to  global  knowledge  resources,  information,  and  interactive  par¬ 
ticipation  in  the  opinion  forming  process  in  the  corporation.  It  should  be  stressed  that 
this  is  true  only  for  potential  users.  Only  those  who  have  access  and  competence  to  use 
and  exploit  this  information  can  derive  benefits  from  these  opportunities.  Cost  effective¬ 
ness  plays  a  crucial  role  in  the  further  development  of  management  in  the  direction  of 
participatory  management. 

There  are  no  clear  perspectives  on  the  future  price  to  use  multimedia  technologies.  Many 
experts  have  the  opinion  that  prices  for  information  transfer  will  drastically  decrease  due 
to  the  complete  liberalization  of  the  telecommunication  market.  They  predict  that  not 
only  the  promotion  of  mass  access  to  technologies,  such  as  the  Internet  and  other  new 
communication  services,  will  be  enforced.  Mass  access  to  information  can  be  an  oppor¬ 
tunity  for  democratic  processes  in  society  and  business.  Right  now  the  information 
highways  are  far  too  slow,  unsafe,  and  too  expensive  for  many  potential  users.  In  many 
companies  the  rules  for  the  Intranet,  the  information  and  data  exchange  within  the  corpo¬ 
ration,  are  just  being  defined.  Whether  or  not  more  participation  can  really  be  achieved 
or  whether  fewer  software  moguls  increase  their  power  remains  to  be  seen. 

Even  today  Bill  Gates  is  known  as  one  of  the  most  influential  entrepreneurs  in  the  world. 
He  is  in  the  position  of  acquiring  the  complete  cultural  inheritance  of  the  world  in  nego¬ 
tiation  with  the  great  museums  of  all  countries  in  the  sense  of  buying  the  right  to  repro¬ 
duce  them.  It  is  expected  that  access  to  Internet  information,  therefore  use  of  Multimedia, 
will  become  more  expensive  the  very  minute  when  the  integration  of  all  business  trans¬ 
actions,  life  experiences  and  educational  processes  of  the  majority  of  people  is  achieved. 
Experiences  in  the  past  show  that  we  will  have  to  expect  additional  fiscal  and  financial 
taxes  in  new  dimensions. 

Meanwhile  powerful  global  multinational  companies  of  information  and  communication 
technologies  and  media  trusts  have  developed  unknown  amounts  of  power  and  political 
influence.  National  governments  cannot  practically  oppose  this  power  with  appropriate 
countermeasures  besides  quarreling,  as  seen  with  TV-transmission  times  and  communi¬ 
cation  channels.  In  this  respect,  there  may  arise  new  dangers  to  democracy.  The  influence 
of  the  national  governments,  as  opposed  by  supranational  organizations,  regional  interest 
groups  and  by  competition  of  private  firms,  will  be  drastically  reduced.  Even  data  relat¬ 
ing  to  individuals,  their  private  consumption,  shopping  habits,  preferences  and  liking 
will  be  ever  more  controlled  by  economic  giants.  The  companies  will  be  obliged  to  store 
and  use  personal  data  according  to  the  technical  state  of  the  art  with  greatest  possible 
care.  This  applies  to  the  use  of  personal  profiles  and  potential  abuse  of  personal  infor¬ 
mation. 
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The  prominent  German  newspaper  FAZ-Magazin  predicts  that  democracy  will  be  the 
great  loser  in  the  information  society  (FAZ-Magazin,  1996).  Opinion  poUs  via  Internet 
will  surely  influence  the  work  of  governments  and  parliaments  and  thus  undermine  their 
ability  to  govern  responsibly.  Each  individual,  who  is  wealthy,  educated  and  has  Internet 
access  becomes  the  winners.  On  the  other  side  are  millions  and  millions  of  illiterates  with 
respect  to  computer  technologies.  Five  biUion  social  losers  currently  have  no  chance  to 
access  the  net.  Members  of  the  industrial  society  look  down  contemptuously  on  “digital 
illiterates“  or  “homeless  in  cyberspace“. 

Even  though  globalization  is  a  major  trend  now,  there  is  also  a  trend  towards  regionali¬ 
zation.  The  “technocratic  digitalos“  and  an  increasing  number  of  people  turn  to  tradi¬ 
tional  structures  to  find  their  identity  in  small  regional  units.  At  the  same  time  there  can 
be  seen  another  trend  towards  isolation  and  crisis  through  the  lack  of  information.  The 
reason  for  this  trend  is  the  power  and  financial  influence  of  managers  and  owners  of  in¬ 
formation.  The  same  trend  is  obvious  for  the  management  and  the  participatory  inclusion 
of  coworkers  in  the  corporation.  The  development  of  democratic  processes  globally,  re¬ 
gionally  in  a  country  and  in  industry  will  be  an  important  prerequisite  in  order  to  take 
countermeasures  against  possible  negative  tendencies  and  to  intensify  the  use  of  positive 
impacts  that  new  technologies  may  offer. 

Opportunities  and  Challenges  for  Multimedia  in  Business 

Technological  innovations  change  products,  production  procedures,  organizations,  mate¬ 
rials,  and  markets.  In  the  clothing  industry  measuring  equipment  will  be  used  increas¬ 
ingly  to  scan  a  human  body  in  three  dimensions  and  to  produce  scans  in  true  color.  These 
scans  are  used  to  show  the  final  result  and  to  cut  the  cloth  appropriately.  A  day  later  the 
customer  will  receive  his  clothes  perfectly  made-to-measure  by  a  direct  delivery  service. 
The  scanned  anthropomorphic  data  will  be  written  to  a  Personal  Body  Card  by  the  cus¬ 
tomer  and  the  salesman  for  later  use.  The  advent  of  high  tech  in  the  world  of  sales  and 
the  textile  industry  requires  new  solutions  from  cyberspace.  The  Textile/Clothing  Tech¬ 
nology  Corporation  (TC**2),  a  200  firm  consortium,  demonstrated  how  a  designer  in 
New  York  cooperates  with  a  customer  using  video-teleconferencing  to  design  clothing 
made-to-measure. 

Today  it  is  not  a  technical  problem  for  the  customer  to  select  fi'om  huge  computerized 
fashion  collection,  the  fashion  pattern,  the  fabric,  colors  and  texture,  even  the  buttons 
and  to  display  the  customer  dressed  in  his  personal  composition  of  clothing  on  a  digital 
photograph.  There  will  arise  a  threatening  competition  for  those  businesses  selling 
clothing  not  made-to-order  and  it  will  be  necessary  for  my  company  to  take  this  new  de¬ 
velopment  of  made-to-measure  clothing  with  great  attention. 

The  philosophy  of  buying  will  play  an  important  role  for  the  customer  buying  his  cloth¬ 
ing.  Shopping  will  be  an  entertainment,  trying  on  and  rummaging  will  be  a  means  of  joy 
and  reduce  efforts  to  find  the  right  clothes  to  a  minimum.  The  greatest  challenges  for 
industry  and  management  resulting  from  closer  links  between  industry,  trade,  and  the 
customer  will  be  soon  visible.  The  processes  in  and  around  enterprise  are  becoming  more 
and  more  complex.  A  special  situation  attained  by  the  direct  connection  that  is  now  pos¬ 
sible  not  only  between  trader  and  customer  but  also  between  producer  and  customer.  For 
many  production  tasks  basic  strategic  questions  arise  for  businesses. 
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Opportunities  for  the  industry  are  in  an  increase  of  efficiency  with  respect  to  internal 
communication  and  informational  processes.  These  opportunities  can  be  enhanced  using 
data  exchange,  videoconferencing,  tele  supply  samplings,  and  education.  Even  external 
communication  is  an  important  future  application  area  of  multimedia.  Offers  can  be 
made  using  online  services,  consumers  can  be  attracted  using  CD-ROM  catalogues,  and 
direct  exchange  of  information  can  be  possible  using  e-mail  or  automatic  exchange  of 
documents.  Decentralized  firms  based  on  telecommuting  have  the  advantage  to  produce 
specific  to  the  customer’s  needs  on  schedule  and  be  cost  efficient.  The  market  position  of 
small  and  medium  firms  can  be  increased  using  state-of-the-art  information  and  commu¬ 
nication  media. 

The  members  of  the  “New  Media“  study  group,  amongst  whose  members  were  delegates 
of  the  Federal  Government  of  Germany,  defined  these  particular  requirements  for  the 
business: 

•  More  customer  orientation  by  increasing  the  quality  of  offers, 

•  Increased  fulfillment  of  global  market  opportunities  (e.g.  by  marketing  using  the 
internet), 

•  Increase  of  the  ability  for  innovation  and  readiness  for  new  services  and  better  per¬ 
formance  orientation  through  more  efficient  use  of  technological  potentials  and  new 
forms  of  cooperation  in  networks, 

•  Adaptation  of  the  corporate  identity  to  new  technological  opportunities  such  as  de¬ 
centralized  organization,  a  stronger  orientation  of  staff  towards  corporate  identity,  re¬ 
duced  number  of  management  levels  in  the  company  management,  lean  and  flat  hier¬ 
archies,  continuing  education  and  qualification  of  staff  in  “learning  enterprises**. 

Currently,  the  highest  priority  use  of  Multimedia  is  within  the  market-oriented  manage¬ 
ment,  a  tendency  called  marketing-communication. 

In  America,  25%  of  the  biggest  corporations  are  represented  on  the  Internet.  Online,  each 
company  has  the  same  market  opportunities.  New  service  providers  can  effectively  com¬ 
pete  against  established  firms,  if  they  are  smart,  innovative  and  offer  their  clientele  inter¬ 
esting  presentations.  Using  multimedia  technologies,  e.g.  the  Internet,  results  in  many 
advantages  for  all  parts  of  a  company  and  many  opportunities  for  the  management: 

•  Time  effective  processing  of  routine  tasks, 

•  Increase  of  productivity  and  at  the  same  time  reduction  of  staff, 

•  More  efficient  use  of  resources  and  time  saving, 

•  Improvement  of  the  basis  of  planning  and  decision  taking  through  better  access  to 
external  information,  early  warning  when  trends  change,  retrieval  of  market  research 
data,  knowledge  of  different  industries  and  economic  trends, 

•  Enhancements  in  production  by  well-aimed  selection  of  suppliers  worldwide,  direct 
communication  with  these  suppliers,  and  online  synchronization  to  reduce  stock, 

•  Changed  aspects  of  corporations  using  global  access  to  electronic  business,  open 
communications  channels,  and  increased  customer  contacts, 

•  More  skillful  product  design  using  closer  connections  to  designers  and  design  con¬ 
sultants,  cooperation  with  research  establishments,  continuous  inclusion  of  new 
global  trends, 

•  Using  expanded  marketing  or  micro-marketing  oriented  strategies  according  to  indi- 


vidual  requiremeiits  and  preferences,  fast  feedback,  market  research  data,  and  con¬ 
tinuous  sales  forecasts  of  trade  companies  worldwide, 

•  Better  customer  orientation,  customer  following  precise  orders  and  deliveries  using 
faster  access  of  skilled  knowledge,  problem  solutions  and  product  information, 

•  New  chances  for  humanization  of  the  world  of  labor, 

•  Improvements  in  the  field  of  personnel  management,  using  electronic  curricula  vitae, 
access  to  differentiated  education,  self-learmng,  information  about  jobs  and  careers 
in  the  corporations,  hiring  of  self  employed  at-home  workers  and  project  oriented 
teams, 

•  More  flexibility  and  at  the  same  time  more  socially  adequate  organization  of  working 
time, 

•  Reformed  information  handling, 

•  Better  and  more  cost  effective  communication,  with  new  potential  customers  world¬ 
wide, 

•  Increase  of  professional  knowledge  offering  chances  for  continuous  exchange  of  ex¬ 
perience, 

and  others. 

Many  corporations  are  starting  to  use  the  opportunities  of  the  Internet  by  offering  to  their 
staff  the  chance  to  use  this  new  medium,  applying  their  own  strategies  and  thus  opening 
the  net  for  important  business  processes. 

As  soon  as  corporations  become  more  flexible  and  smaller,  the  importance  of  alliances 
for  the  success  of  the  corporation  will  rise.  Information  management  will  develop  as  a 
key  factor.  Everybody  who  wants  to  initiate  alliances  successfully  needs  frictionless, 
properly  timed  and  unhindered  flow  of  information  and  communication.  Corporations 
require  international  presence,  “global  sourcing“.  They  also  need  development  teams  to 
cooperate  worldwide  using  time  differences  to  be  innovative  and  creative  24  hours  a  day. 
There  is  an  urgent  need  to  define  and  optimize  management  information  and  information 
structures  that  are  crucial  for  the  business  purpose  and  to  reform  process  and  informa¬ 
tional  structures  as  well  as  data  architecture.  To  guarantee  efficiency  in  management  of 
information  there  are  further  requirements  than  simply  technological  competence.  Virtu¬ 
osity  will  be  required  for  those  who  find  synergies  in  knowledge  resources  with  respect  to 
a  specific  problem  to  derive  innovative  developments  for  the  corporation. 

This  means  drastic  changes  for  the  top  management  as  well  as  the  employees.  It  also  re¬ 
quires  continuous  learning  and  a  lot  of  intuitive  selecting  of  information.  The  Internet  is 
not  only  a  source  of  relevant  information  but  also  a  huge  ocean  of  banahties  with  singu¬ 
larities  of  great  value  (Martin,  1997b). 

Using  Multimedia  on  the  level  of  a  corporation  is  more  crucial  to  solving  application 
problems  than  to  technical  aspects.  In  this  respect,  there  are  numerous  new  challenges  for 
companies.  Requirements  relate  to  the  development  of  worldwide  new  information 
sources,  but  also  in  designing  a  network  of  development  and  production  stages.  New 
virtual  structures  in  organization  and  labor  will  develop  in  combination  with  new  indus¬ 
trial  processes  that  today  cannot  even  be  imagined.  Integrated  document  management, 
diagnosis  of  technical  systems  using  remote  access  and  analysis  systems  will  be  even 
more  effective  using  many  new  internal  applications.  Even  now,  there  is  definitely  a  fast 
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reduction  of  hierarchies  and  a  much  faster  flow  of  information  using  multimedia  tech¬ 
niques,  Management  can  be  developed  more  efficiently  and  in  a  more  process-oriented 
manner  offering  completely  new  possibilities  in  connection  with  multimedia  customer 
service.  Not  only  the  introduction  of  computer  oriented  manufacturing,  but  especially 
computer  supported,  customer  oriented,  and  interactive  mass  production-to-measure,  will 
lead  to  the  establishment  of  numerous  new  task  areas.  These  applications  within  the  cor¬ 
poration  require  permanent  learning  processes  within  industry.  At  the  same  time  the  de¬ 
velopment  of  intra-networks  and  changes  in  the  overall  communication  structure  require 
new  efforts.  The  aim  is  to  react  in  a  timely,  quality  oriented,  and  predetermined  way  that 
is  flexible  to  customer  requirements.  At  the  same  time  customers  should  be  included  in 
interactive  design  processes.  This  idea  is  already  in  preparation  in  the  field  of  industrial 
mass  confection  made-to-measure.  We  can  predict,  especially  in  this  professional  area, 
most  of  the  revolutionary  consequences.  At  the  same  time  there  are  new  problems  with 
security.  These  problems  do  not  only  relate  to  the  increasing  use  of  virtual  money  and 
digital  signatures  but  also  to  new  dangers  of  industry  espionage  and  the  potential  break¬ 
down  of  complete  information  systems  and  databases.  In  response  to  this  I  would  like  to 
study  the  problem  of  computer  viruses  and  hackers.  I  would  also  like  to  study  the  prob¬ 
lems  of  the  intelligent  theft  encouraged  by  high  tech  systems  that  are  not  completely  safe, 
especially  since  our  industry  is  one  of  the  most  affected  branches. 

Changes  in  Management  Processes  due  to  Communication  and  Information  Tech¬ 
nologies 

The  possibilities  for  representing  corporations  and  products  on  the  Internet  are  plentiful 
and  diverse.  Company  information  and  brands  can  be  displayed  and  put  forth.  There  are 
advertisements  for  many  brands  including  promotional  actions  such  as  sponsored  events, 
games,  and  price  opportunities  to  attract  customers.  New  ways  of  promotion  are  being 
developed  using  virtual  showcases.  Multimedia  shops,  video  entertainment  and  shop 
radio.  Shopping  will  become  an  “adventure  joumey“,  parts  of  the  selection  will  be  con¬ 
nected  to  product  worlds  or  can  be  reorganized  in  new  worlds  by  the  customer  himself. 
The  Internet  has  received  a  degree  of  user-friendliness  by  its  new  graphical  user  inter¬ 
face,  the  World  Wide  Web. 

Entrepreneurs  and  top  management  have  to  ask  themselves  whether  the  corporation  shall 
be  developed  to  a  virtual  corporation,  a  cyber-enterprise,  and  whether  it  is  possible.  They 
will  also  need  to  decide  whether  there  is  a  profound  strategic  vision  and  whether  there  are 
qualifications  and  skills  -  or  competence  in  the  enterprise  altogether-  to  face  these  new 
challenges. 

New  developments  in  information  and  communication  technologies  result  in  necessary 
changes  for  the  corporation  such  as: 

•  Fast  reaction  to  events  and  -  if  necessary-  to  take  a  favorable  turn  into  a  new  direc¬ 
tion  of  corporate  strategy, 

•  Continuous  improvement  of  products  and  services  and  introduction  of  new  products, 

•  Ongoing  development  of  applied  strategies  to  permanently  enhance  and  change 
products. 
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speed  of  reaction  in  many  situations  will  become  a  crucial  factor  in  competition  for. 

•  Recognizing  requirements  of  new  products, 

•  Bringing  new  products  to  market, 

•  Establishing  new  services, 

•  Improving  products, 

•  Reacting  to  trends  in  fashion, 

•  Satisfying  customer  requirements, 

•  Controlling  stock  and  distribution  of  goods. 

Technical  information  and  communication  processes  will  put  forth  a  significant  contri¬ 
bution  in  these  areas.  We  must  be  even  more  conscious  of  die  fact  that  human  communi¬ 
cation  is  a  decisive  part  of  our  culture  that  will  be  critically  influenced  especially  by  the 
new  information  and  communication  techniques  and  the  use  of  multimedia.  Using  the 
new  media  there  is  a  completely  different  form  of  communication  between  individuals. 
The  digital  age  is  understood  by  many  people  as  birth  of  a  new  confidence  that  is  based 
on  digital  principles.  The  completely  new  cultural  aspects  of  life  in  our  so  called  “global 
village^,  the  new  Internet  culture,  has  not  yet  been  researched  sufficiently  as  of  yet.  In 
the  “First  Annual  Report  to  the  European  Commission  from  the  Information  Society  Fo- 
rum“  of  June  1996  there  was  an  introduction  of  128  members  of  the  Forum  who  were 
elected  by  the  commission.  Their  background  can  be  subsumed  under  five  groups: 

•  users  of  the  new  technologies:  industry  (banks,  retail,  maritime  etc.),  public  services, 
consumer  groups,  small  and  medium- sized  enterprises  and  the  professions 

•  social  groups:  academics,  employers  organizations,  trade  unions,  youth  groups,  re¬ 
gional  and  city  representatives 

•  content  and  service  providers:  publishers  and  authors,  film  and  TV  producers, 
broadcasters,  computer  software  producers  and  information  service  providers 

•  network  operators:  telecommunications  repair  technicians,  cable  TV,  mobile  and 
satellite  operators 

•  Institutions:  members  for  Parliament,  of  the  Economic  and  Social  Committee  of  the 
Committee  of  the  Regions  and  the  data  protection  Commissioner. 

In  six  working  groups  they  dealt  with  the  following  questions 

•  the  impact  on  the  economy  and  employment 

•  basic  social  and  democratic  values  in  the  „virtual  community** 

•  the  influence  on  public  services 

•  education,  training  and  learning  in  the  Information  Society 

•  the  cultural  dimension  and  the  future  of  the  media 

•  sustainable  development,  technology  and  infrastructure 

These  tasks  are  probably  correct.  Yet  it  is  not  clear  whether  they  address  the  really  deci¬ 
sive  question  in  appropriate  depth:  the  evaluation  of  information  and  knowledge. 

Filtering  the  growing  flood  of  information  individually  and  converting  information  into 
applicable  useful  knowledge  is,  from  my  point  of  view,  the  primary  purpose  and  presents 
the  greatest  challenge  for  today’s  managers.  New  multimedia  systems  should  not  only 
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offer  information,  but  also  explain  complex  contexts  in  an  acceptable  manner.  At  the 
same  time  any  information  of  any  content  should  be  able  to  be  produced  and  offered 
globally. 

From  my  understanding  it  is  a  fatal  error  to  believe  that  more  information  also  improves 
the  quality  of  information.  We  all  know  from  our  practical  experience  that  more  infor¬ 
mation  guarantees  by  no  means  a  qualitatively  new  level  of  being  informed.  Even  in  the 
information  age  we  must  not  oversee  that  useless  information,  redundant  information, 
false  information,  misinformation,  and  catastrophic  information  serve  no  purpose  as 
people  are  frequently  flooded  by  data  and  are  still  hungry  for  the  right  information. 
Evaluation  of  information  is  therefore  frequently  more  important  than  the  information 
itself.  This  subject  is  a  task  that  cannot  be  easily  formalized  on  the  corporate  level.  In 
how  far  multimedia  information  processing  enhances  and  facilitates  the  adoption  of 
knowledge  by  offering  a  better  emotional  access  to  our  brain  or  whether  it  hampers 
knowledge  acquisition  by  sacrificing  resources  to  redundant  information,  will  be  a  ques¬ 
tion  of  continuous  learning  within  the  corporation. 

Consequences  of  Multimedia  for  Employment 

The  political  consequences  with  respect  to  the  task  of  new  multimedia  developments  are 
many  and  complex.  There  is  hardly  an  industry  that  wiU  not  be  influenced  by  new  tech¬ 
nical  innovations.  The  change  of  existing  professions  and  the  development  of  new  ones 
will  determine  the  organization  of  work.  It  will  also  be  determined  by  changes  of  forms 
of  cooperation  within  the  corporations,  between  corporations,  and  in  industiy.  Neverthe¬ 
less,  the  potential  for  improved  labor  efficiency  and  effectiveness  is  highly  controversial. 
One  group  understands  Multimedia  as  the  greatest  hope  for  new  workplaces,  others  de¬ 
fine  it  as  a  “job-killer“.  As  seen  historically  both  sides  are  correct.  Increases  in  produc¬ 
tion  have  lead  to  more  growth  and  new  jobs  in  the  long  run.  Yet  it  is  not  clear  how  long  it 
will  take  until  production  relevant  increases  of  incomes  will  lead  to  a  higher  request  for 
goods  and  as  such  compensate  for  prior  effects  of  rationalization.  Nobody  can  teU  today 
which  and  where  new  products  and  services  will  develop  and  lead  to  a  higher  level  of 
employment,  Fourastid  (Fourasti6,  1954)  based  his  theories  on  his  so  called  3-sector- 
model  in  which  he  predicted  employment  and  technical  development  in  agriculture  and 
forestry,  industry,  and  services  related  to  employment  as  10:10:80.  Initiated  by  the 
OECD  in  the  70’ s,  there  was  a  reorganization  of  statistical  measures  taking  a  fourth  sec¬ 
tor,  the  information  sector,  into  account.  This  sector  was  defined  using  a  very  wide 
scope.  It  included  all  administrational  professions,  the  complete  educational  world,  even 
foresters  and  inn  keepers,  and  other  “information  processing“  professions. 

Multimedia  inspires  the  imagination  of  many  but  for  most  of  it  is  still  in  a  state  of  ex¬ 
perimentation.  The  most  important  purpose  of  the  new  information  and  communication 
technologies  is  to  divert  routine  jobs  to  machines  and  then  eliminate  them.  Computer 
technologies  serve  the  purpose  of  increasing  efficiency  and  quality  in  the  corporation. 
Unskilled  typists  and  low  ranking  managers  who  only  conduct  routine  tasks  can  easily  be 
replaced  by  cheap  and  reliable  software  products.  The  consequence  of  this  is  fear  because 
finding  a  new  job  within  the  corporation  is  not  likely.  Everywhere,  in  production,  in  ad¬ 
ministration,  or  in  the  office,  knowledge  is  used  productively  in  a  systematic  way.  Op¬ 
portunities  for  effecting  change  are  used  in  an  optimal  way,  and  cost  will  be  reduced. 
Many  experts  talk  about  a  new  ethic  of  efficiency  that  is  applied  to  all  branches  of  the 
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business  and  which  are  directly  or  indirectly  linked  with  modem  information-  and  com¬ 
munication  technology. 

Studies  of  the  University  of  Wurzburg,  Germany,  prove  that  more  than  6.7  million  jobs 
of  the  current  22  million  jobs  in  this  country  could  be  cleared  away  by  introducing  the 
new  information  and  communication  technology.  One  example  for  the  amount  of  poten¬ 
tial  workplace  reduction  is  set  out  in  the  following  table: 


Professional  Domain 

Employees  1996 

(not  part  of  the  study) 

Estimated  unemployed 

people 

(after  introduction  of  IT 

technologies) 

Banking 

771.667 

473.718 

Consulting 

844.000 

293.200 

Education 

914.233 

249.426 

Office 

1.465.000 

806.000 

Health  Services 

839.192 

293.941 

Conunerce 

3.382.192 

1.726,961 

Public  Administration 

2.604.000 

1.199.769 

Planning 

194.000 

63.950 

Cleaning 

835.000 

138.150 

Others 

1.609.000 

304.430 

Transport/Logistics 

897.100 

666.860 

Renting 

236.000 

118.030 

Insurance 

660.400 

390.400 

Sales  Promotion 

80,000 

15.150 

Total 

15.331,784 

6.739.985 

This  relates  especially  to  branches  such  as  banks  and  insurance  companies,  consulting, 
health  services,  commerce,  public  services,  transport,  logistics,  and  simple  service  pro¬ 
fessions  such  as  in  renting  offices,  which  are  subject  to  rationalization  even  more.  The 
already  mentioned  “First  Annual  Report  to  the  European  Commission  from  the  Informa¬ 
tion  Society  Fomm“  underlines  the  findings  firom  the  point  of  view  of  the  consequences 
with  respect  to  employment  using  the  following  thesis: 
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“In  order  to  maximize  the  job-creating  benefits  of  the  new  technologies  as  quickly  as 
possible,  it  is  essential  and  urgent  for  companies  and  organizations  of  all  sizes  to  adapt 
their  organizations  and  structures.  Until  this  process  is  well  underway,  the  Information 
Society  looks  likely  to  destroy  more  jobs  than  it  creates”. 

This  strengthens  the  so-called  “job-kilhng-thesis“. 

In  the  future  the  adapted  principle  of  lifelong  employment  will  be  an  unrealistic  ideal. 
Outsourcing  of  processes  in  enterprise  will  increase.  More  and  more  work  will  be  con¬ 
verted  to  self  employed,  free  lancing  and  temporary  jobs.  A  new  highly  competitive  mar¬ 
ket  will  develop.  Not  only  will  telecommuting  become  more  popular,  but  also  improved 
flows  of  information  will  make  work  processes  more  flexible.  It  will  become  easier  for 
firms  to  acquire  external  workers.  The  consequence  will  be  that  the  corporations  can 
work  more  effectively,  be  more  decentralized,  and  can  decrease  in  size.  Telecommuting, 
teleservice,  telecooperation,  telecommunication,  and  even  telefeeling  will  no  doubt  re¬ 
duce  personal  and  direct  human  contact  even  more.  It  will  depend  a  great  deal  on  the 
management  how  far  the  personality  of  a  human  employee  and  his  chance  for  creativity 
and  self-development  will  be  dealt  with.  And  it  will  also  depend  on  the  management  for 
which  positive  or  negative  consequences  the  increasing  number  of  virtual  enterprises  will 
bring  about,  that  can  be  transferred  from  one  location  to  another  and  easily  seize  and 
start  operation. 

A  study  of  the  German  Institute  for  Work  and  Technology  ( Institut  fUr  Arbeit  und  Tech- 
nik,  1997)  shows  that  telecommuting  or  telework  has  a  growing  importance  in  combina¬ 
tion  with  major  shortcomings.  Especially  for  women  telecommuting  offer  new  job  op¬ 
portunities  but  it  does  result  in  some  disdvantages.  Housework  and  professional  work 
have  to  be  performed  at  the  same  time  increasing  workload.  Telecommuting  also  tends  to 
reduce  chances  of  a  career  because  career  is  closely  linked  to  presence  in  the  enterprise. 

Now  I  would  like  to  talk  about  another  consequence  of  Multimedia  with  respect  to  labor, 
which  is  the  global  competition  for  the  company  location.  This  is  not  only  a  competition 
between  products  and  services  but  also  a  struggle  for  work  places.  Self-employed  persons 
must  be  prepared  for  an  intensified  competition  with  respect  to  accomplishment  and 
price  for  their  products.  Even  developing  countries  possess  highly  specialized  experts, 
who  can  access  global  markets  without  restrictions.  From  all  these  examples  it  is  clear 
that  the  concern  about  the  degree  of  employment  is  very  justified. 

After  talking  in  detail  about  the  negative  aspects  of  the  employment  situation  I  would 
like  to  contribute  to  the  position  of  “Promising  Multimedia“.  We  must  not  forget  the 
positive  employment  tendencies.  These  tendencies  promise  to  maintain  employment  and 
even  define  new  workplaces  using  increased  efficiency.  Eveiything  depends  on  the  abil¬ 
ity  of  structural  change  in  the  corporations  and  the  position  in  the  international  competi¬ 
tion,  The  optimistic  attitude  of  many  is  based  on  the  macroeconomic  assumption,  that 
earnings  with  respect  to  productivity  cannot  simply  be  ignored  with  respect  to  employ¬ 
ment.  Increased  buying  power  in  different  areas  will  strengthen  demand  and  help  to  de¬ 
velop  new  workplaces.  But  it  is  only  speculation  as  to  what  extent  this  will  happen. 
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Another  thesis  in  the  first  annual  report  is  the  following: 

„Teleconimuting  offers  many  job-creating  possibilities  and  attractive  improvements  in 
working  lifestyles,  although,  it  raises  many  important  issues  for  labor  laws  and  collective 
bargaining.  Public  policy  must  facilitate,  not  obstruct,  the  development  of  teleconunut- 
ing.  Promoting  a  mix  of  home  and  office  telecommuting  is  an  effective  way  of  handling 
fears  that  telecommuting  exclusively  in  the  home  may  be  too  isolating. “ 

These  sentences  make  clear  that  even  the  protagonists  of  Multimedia  cannot  contribute 
concretely  to  the  blessings  of  new  information  and  communication  technologies. 

The  brief  effect  of  rationahzation  cannot  be  avoided  because  the  corporations  are  forced 
to  produce  as  cost  effectively  and  efficiently  as  possible  to  withstand  international  com¬ 
petition.  Corporations  must  reduce  product  cycles  and  at  the  same  time  increase  the  vari¬ 
ety  of  products  and  models.  They  must  also  reduce  development  and  delivery  times,  in¬ 
crease  product  quality  and  invent  customer  oriented  system  solutions.  Currently,  there  is 
no  reasonable  alternative  to  the  new  technologies,  or  else  all  workplaces  would  be  endan¬ 
gered.  In  how  far  information  technologies  are  used  for  the  development  of  new  products 
or  to  improve  these  products  is  also  dependent  on  management  and  motivation  as  well  as 
skill  of  the  employees.  Discovery  of  new  problem  areas  will  be  an  important  challenge 
for  the  management. 

To  guarantee  a  good  future  standing  of  Germany  as  a  major  industrial  player,  the  already 
mentioned  working  group  ‘'New  Media"  was  founded.  In  this  working  group  delegates  of 
the  German  Federal  Government  and  those  organizations  were  represented  that  are 
"members  of  the  inner  circle  of  the  Bundeskanzler",  the  Chancellor  of  the  Federal  Re- 
pubhc  of  Germany.  In  this  working  group  positive  influences  with  respect  to  the  level  of 
employment  were  identified,  especially  in  the  information  and  communication  markets. 
Despite  the  uncertainty  of  the  prognoses  in  a  series  of  different  studies,  quantitative  ef¬ 
fects  of  employment  were  analyzed  that  could  be  initiated  using  new  media.  Results  of 
an  analysis  of  Prognos/DIW  in  Germany  came  to  the  conclusion  that  about  180.000 
workplaces  in  economical  branches  could  develop  that  belong  to  the  media-  and  commu¬ 
nication  sector  until  the  year  2010.  The  highest  potentials  are  with  so  called  “content- 
providers"  and  in  the  area  of  computer  software  as  well  as  computer  services.  A  smdy  of 
the  METIER-  consortium  for  the  European  Union  calculates  that  by  expansion  of  new 
information  and  conununication  technologies  up  by  year  2010  six  million  additional 
workplaces  could  be  generated.  From  these  six  million  workplaces  -on  a  calculated  ba¬ 
sis-  1.5  milhon  could  be  established  in  Germany.  Such  estimates  suffer  from  a  lot  of 
methodological  problems.  As  there  are  no  concrete  prognoses  at  the  present  time,  we 
should  concentrate  on  the  fact  that  only  the  creation  of  mass  markets  will  develop  em¬ 
ployment  effectively.  The  potential  of  technical  innovations  can  only  be  fully  utilized  if 
they  are  accompanied  by  complementary  organizational  changes  in  work  and  production. 
Information  society  alone  cannot  solve  the  problem  of  employment.  The  solution  of  that 
problem  will  by  no  means  be  possible  without  lifelong  education  and  social  innovation. 

Electronic  Shopping  and  Cybercash  Unlimited 

A  completely  new  challenge  for  the  future  is  the  increasing  use  of  electronic  shopping  for 
all  enterprises.  All  business  transactions  will  radically  change,  not  only  from  use  of  elec- 


25 


tronic  commerce  and  interactive  data  exchange,  but  even  more  by  new  digital  worldwide 
shopping  possibilities. 

Buying  habits  will  drastically  change.  We  do  not  yet  know  which  will  be  the  “killer  ap- 
plications“  of  information  society.  To  identify  these  killer  applications,  pilot  projects 
should  be  conducted  in  sample  population  as  much  as  possible.  Not  only  the  technical 
aspects  of  transaction  must  be  tested,  but  also  the  readiness  of  the  customer  to  buy  and  to 
pay. 

What  will  the  achievements  of  online  shopping  be?  Once  a  customer  has  developed  a 
need  for  a  particular  item  they  may  search  online  for  that  item  that  best  matches  the  price 
range  and  specifications  they  desire.  The  order  could  then  be  taken  and  acknowledged 
online.  The  customer  would  receive  immediate  acknowledgement  of  purchase.  Using 
direct  data  entry  checked  by  the  customers  themselves,  transaction  errors  could  be  re¬ 
duced  and  orders  can  be  processed  in  a  cost-effective  way.  The  customer  could  make 
purchases  easily  and  efficiently,  without  being  restricted  by  business  hours.  Buying  is 
made  virtually  stress  fi*ee.  In  this  way  the  customers  needs  are  best  served. 

Presently  single  enterprises  in  industry  and  commerce  estimate  the  future  of  multimedia 
shopping  very  differently.  Still  it  is  very  probable  that  radical  changes  of  the  structure  of 
commerce  will  happen  in  the  next  decade.  This  change  requires  a  timely  orientation  to¬ 
wards  these  new  buying  procedures.  The  basic  credo  of  the  information  society  is  that 
everything  is  available  everywhere  and  any  time. 

Home  shopping  is  not  new  to  people.  Mail  order  houses  have  guaranteed  comfortable 
shopping  for  a  long  time.  Yet,  teleshopping  will  fundamentally  change  the  presentation 
and  display  of  the  whole  portfolio  of  goods.  Buying  will  be  possible  24  hours  a  day  and 
worldwide.  The  inherent  loss  of  sensual  pleasure  especially  in  fashion  will  establish  some 
barriers  but  not  change  the  basic  trend.  Until  recently  it  could  not  be  estimated  what  con¬ 
sequences  this  would  have  on  small  retail  establishments  or  the  larger  commercial 
chains.  However,  with  the  development  of  online  shopping,  many  chances  emerge  for 
commerce  to  manipulate  consumer  behavior.  Individual  shopping  habits  can  be  recorded, 
analyzed,  and  stored  in  databases  able  to  generate  complete  buyer  profiles  for  use  in  sub¬ 
sequent  marketing  decisions.  With  regards  to  global  competition,  borders  between  pro¬ 
ducers  and  commerce  will  change  under  the  influence  of  these  new  technologies.  Many 
producers  plan  to  utilize  direct  sales,  factory  outlets,  and  new  forms  of  electronic  ware¬ 
housing  to  boost  sales. 

New  procedures  in  management  such  as  efficient  “consumer  response“  or  “category 
management“  are  predecessors  of  this  development.  International  regulations  on  product 
liability,  advertisement,  and  consumer  protection  are  becoming  more  and  more  necessary 
in  a  world  of  increasing  electronic  shopping. 

Multimedia  and  Simulation:  Human  Machine  Interface  Implications 
There  is  still  another  facet  of  Multimedia  that  I  have  not  yet  discussed.  This  aspect  of 
Multimedia  does  not  utilize  this  technology  as  an  instrument  for  shopping  or  as  a  tool  for 
today’s  managers  to  operate  their  enterprise.  Instead,  it  is  the  aspect  of  Multimedia  that 
makes  work  at  a  complex  technical  plant  acceptable,  makes  modem  systems  controllable 
and  guarantees  safe  and  efficient  operation  of  complex  systems.  This  facet  includes  the 
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aspects  of  leisurely  use,  entertainment,  and  both  useful  and  joyful  implications  for  hu¬ 
mans. 

Many  aspects  of  our  everyday  life  can  only  be  accommodated  through  the  use  of  techni¬ 
cal  support.  As  those  supporting  systems  become  more  and  more  sophisticated,  they  will 
require  increasingly  complex  technical  equipment.  More  complex  technical  systems  re¬ 
quire  an  increase  in  the  skills  of  the  human  operator  to  guarantee  their  safe  and  efficient 
use.  Many  studies  with  respect  to  reliability  of  human  operators  prove  the  great  influence 
of  the  human  factor.  Reports  of  commissions  to  clear  up  accidents  and  malfunctions  of 
complex  technical  systems  contribute  a  great  deal  to  these  studies.  Later  in  this  chapter 
F 11  try  to  develop  a  systematic  approach  to  clarify  in  what  areas  of  life  and  work  such 
technical  support  is  undeniable  and  something  nobody  should  do  without.  More  and 
more  sophisticated  technical  support  thus  calls  for  a  carefully  optimized  control  station 
to  reliably  operate  complex  systems.  As  this  optimization  is  not  just  a  technical  optimi¬ 
zation,  but  has  to  include  the  human  operator  as  well,  it  will  be  an  iterative  design  proc¬ 
ess  comprising  both  the  technical  system  components  and  the  human  factor.  The  human 
factor  can  be  optimized  by  developing  refined  and  adapted  methods  of  training.  Thus 
even  a  sub-optimal  system  can  still  be  controllable.  The  far  better  approach,  though,  is 
the  adaptation  of  the  technical  system  to  the  requirements  and  characteristics  of  the  hu¬ 
man  operator. 

There  is  a  new  term  called  “Paradox  of  the  Information  Technology”  (Brddner,  1997). 
This  paradox  explains  the  divergence  of  technical  functions  and  human-machine  per¬ 
formance.  The  more  complex  technical  systems  are,  the  more  performance  would  be  ex¬ 
pected.  Frequently  the  contrary  is  happening.  Making  users  believe  that  technical  sys¬ 
tems  are  intelligent  may  result  in  inadequate  confidence  into  those  systems.  Therefore  it 
is  crucially  important  that  complex  systems  explain  the  way  of  their  decision  making  to 
the  human  operator.  Here  again  an  important  aspect  of  the  human -machine  interface  is 
described. 

Adaptation  of  the  system  to  man  requires  a  careful  analysis  of  human  abilities  and  limi¬ 
tations,  Multimedia  is  a  more  recent  approach  to  displaying  information  at  a  control  sta¬ 
tion  in  a  more  acceptable  and  multisensual  manner.  It  is  thus  better  suited  to  inform  the 
responsible  person(s)  about  the  state  of  the  technical  system.  Multimedia  can  also  be  a 
technology  to  broaden  the  channel  of  operator  input  to  the  system  and  thus  facilitate  in¬ 
teraction  between  human  and  technical  systems  components. 

There  is  a  new  term  for  the  interaction  of  complex  technical  systems  using  Multimedia 
support  at  a  control  station  for  safe  and  efficient  operation.  This  term  is  the  “Human  Ma¬ 
chine  Interface“  for  the  control  of  human  machine  systems.  As  already  mentioned,  opti¬ 
mized  Human  Machine  Interfaces  will  use  techniques  available  to  make  system  operation 
more  efficient,  safer,  and  more  acceptable  to  human  operators.  Traditional  interfaces 
predominantly  use  the  human  visual  sense  for  information  display.  This  is  the  historical 
technique  because  craftsmen  have  during  all  times  in  history  judged  their  work  processes 
by  looking  at  their  work.  Today  the  display  is  mostly  an  electronic  display,  a  screen  dis¬ 
play  offering  graphics  output  rather  than  columns  of  figures.  There  are  cyber  helmets 
available  today  that  literally  offer  a  far  closer  view  of  remote  or  simulated  environments. 
Using  these  Multimedia  system  components,  a  deeper  navigation  in  the  world  of  the 
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Human  Machine  Interface  is  supported,  A  new  term  to  describe  these  aspects  of  human 
control  of  technical  systems  is  “immersion^  into  the  controlled  process  with  the  conse¬ 
quence  of  an  ever  more  perfected  “telepresence*'  wherever  required.  Many  modem  ap¬ 
proaches  offer  auditory  information  in  addition  to  visual  data.  From  simple  warmng 
horns  in  the  past,  to  stereo  headsets  for  pilots  offering  voice  warnings  from  the  direction 
in  which  a  malfunction  is  located.  Vestibular  clues  are  presented  to  pilots  in  a  moving 
base  flight  simulator  enhancing  the  fidelity  of  operation.  There  are  even  approaches  us¬ 
ing  the  haptic  and  olfactory  senses  of  man.  These  multimedia  information  displays  in 
conjunction  with  the  data  glove  and  other  non  traditional  control  devices  complete  the 
instrumentation  of  modem  Human  Machine  Interfaces  optimized  from  an  ergonomic 
viewpoint  operating  with  the  support  of  Multimedia. 

Multimedia  and  the  Human  Machine  Interface  are  tightly  hnked.  The  optimization  proc¬ 
ess  is  a  topic  for  research  establishments,  universities,  and  ergonomic  departments  in 
industry  to  investigate.  It  is  an  interdisciplinary  science  including  the  disciplines  of  engi¬ 
neering,  psychology,  and  work  medicine.  National  and  international  organizations  and 
associations  are  working  in  the  field  of  Human  Engineering  or  Human  Factors  Engi¬ 
neering  in  the  research,  development,  and  application  as  well  as  optimization  and 
evaluation  of  Human  Machine  Interfaces.  Beyond  these  are  the  GeseUschaft  fur  Arbeit- 
wissenschaft  in  Germany,  the  Ergonomics  Society  of  Great  Britain,  the  International  Er¬ 
gonomics  Association  (IE A)  and  the  Human  Factors  and  Ergonomics  Society  in  the 
United  States  (HFES).  For  example,  the  HFES,  more  specifically  its  European  Section, 
is  organizing  a  conference  on  exactly  the  topic  of  ‘Multimedia  and  Simulation.  Human 
Machine  Interface  Implications"  in  my  city  Bochum,  Germany,  in  November  1997 
(Holzhausen,  1997). 

Now  we  know  that  Human  Machine  Interfaces  are  crucial  to  the  interaction  between  a 
human  and  a  complex  technical  system  respectively  its  underlying  process.  Yet,  multi- 
media  Human  Machine  Interfaces  do  not  only  serve  the  purpose  of  controlling  “real" 
physical  processes.  Frequently  they  are  used  to  control  “simulated"  processes.  Many 
people  do  not  realize  this,  but  simulations  are  very  important  today  and  very  helpful.  Ex¬ 
perts  can  “model"  a  process  even  better  if  they  know  more  about  the  process  and  describe 
it  in  a  mathematical  way.  In  this  way  the  parameters  influencing  the  processes  will  be 
recognized.  The  effect  of  these  parameters  and  their  changes  on  the  process  can  then  be 
determined.  Important  examples  for  such  complex  processes  are  the  bloodflow  in  the 
human  body,  meteorology,  or  aerodynamic  aircraft  models. 

Researchers,  engineers  and  other  experts  can  “use"  the  simulated  system  much  like  the 
real  one,  try  to  improve  the  model  and  afterwards  use  it  for  diagnosis,  weather  forecast  or 
as  basis  for  constructing  a  plane.  Sometimes  a  simulation  will  be  run  at  the  same  control 
station  that  is  used  to  control  the  real  process.  One  such  example  would  be  a  traffic 
simulator  connected  with  a  real  control  system  controlling  traffic  lights  in  a  city  in  order 
to  improve  the  community  traffic  flow. 

One  important  criterion  in  a  modern  human  engineered  multimedia  control  station  is  the 
“factor  time".  The  factor  time  can  be  very  efficiently  used  to  increase  system  perform¬ 
ance,  and  safety.  Real  processes  perform  in  “real  time".  All  system  components  including 
the  human  operator  must  react  in  the  real  time  frame.  Simulation  though  can  use  time 
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compression  methods  effectively  to  arrive  at  simulation  results  much  faster  than  the  real 
system.  Meteorologists  compute  weather  models  independently  of  the  real  time,  produce 
a  weather  forecast  much  faster,  and  distribute  a  gale  warning  in  time.  A  simulation  can 
also  be  conducted  in  “slow  motion^.  This  guarantees  a  much  more  detailed  view  for  the 
human  trying  to  understand  turbulent  effects  of  air  behind  an  aircraft  wing  or  in  a  blood 
vessel,  especially  if  the  Human  Machine  Interface  offers  adequate  Multimedia  support. 

I  hope  I  could  make  clear  that  Human  Engineering  and  Multimedia  promise  great  sup¬ 
port  to  humans  in  a  complicated  world  if  it  is  properly  designed  and  correctly  applied. 

Let  me  try  to  describe  possible  application  areas  to  Multimedia  in  connection  with  Hu¬ 
man  Machine  Interfaces  through  a  short  systematic  approach.  A  lot  of  applications  exist 
in  traffic  systems  on  land,  in  the  water  and  in  aeronautical  applications.  Many  cockpit 
systems  in  modern  cars,  ships,  and  aircraft  use  multimedia  support  for  navigation,  diag¬ 
nosis,  and  communication.  The  same  is  true  in  aerospace  applications.  The  space  shuttle 
requires  many  human  engineered  control  systems  to  safeguard  its  operation. 

Stationary  control  centers  use  Multimedia  too.  Examples  are  control  of  traffic  systems, 
police  forces  or  fire  brigades,  air  traffic  control  and  space  systems.  Operations  of  a  space 
station,  the  Mars  Rover,  a  Venus  rocket,  or  the  Hubble  Telescope  are  unthinkable  with¬ 
out  simulation  and  Multimedia  in  the  control  room. 

In  logistics  ever  increasing  demands  require  Multimedia  for  applications,  such  as  man¬ 
agement  of  truck  fleets,  the  storage  of  goods  in  a  warehouse  or  worldwide  supply.  Indus¬ 
try  depends  on  Multimedia  in  aU  the  well  known  “Computer  Aided  Technologies*'  such 
as  CAD,  CAE,  CAD,  and  CAM  or  process  control  in  a  plant. 

As  already  mentioned  modem  medicine  makes  use  of  Multimedia  in  diagnosis  (Ultra¬ 
sonic,  tomography,  data  transmission  between  clinics  and  telepresentation  of  X-ray  pho¬ 
tographs),  3D  man  models  and  stereotactics  in  surgery  and  3D  endoscopy. 

In  research  and  development  literature  reviews  using  hypertext  and  hypermedia  support 
are  state  of  the  art.  Teleleaming  and  Videoconferencing  are  widely  adopted.  New  didac¬ 
tic  concepts  are  being  developed  and  applied  in  universities,  home  learning.  Even  virtual 
universities  are  coming  into  existence. 

In  architecture,  applications  such  as  urban  development,  reconstruction  of  antique  sites, 
virtually  real  buildings  are  widely  adopted.  Navigation  through  such  buildings  as  the 
famous  Frauenkirche  of  Dresden,  which  was  destroyed  in  World  War  n  and  now  is  under 
reconstmction,  are  possible. 

Terrain  models  for  ecological  protection,  seismographic  measuring  campaigns,  or  satel¬ 
lite  image  processing  require  Multimedia  and  human  machine  interface  optimization. 

Last  but  not  least  advertisement  and  entertainment  are  mass  markets  under  heavy  pres¬ 
sure  and  competition.  Virtual  reality,  virtual  actors,  and  tools  for  sophisticated  merging 
of  movies  and  computer  graphics  animation  are  practical  examples  for  near  perfect  pres¬ 
entation  of  Multimedia  techniques  to  humans. 
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What  are  the  consequences  and  benefits  of  applying  Multimedia  to  the  design  and  op¬ 
eration  of  Human  Machine  Interfaces?  First  of  all  safety  aspects  require  every  possible 
effort.  These  safety  aspects  include  both  work  safety  with  respect  to  employees  and  to  the 
population  living  in  the  area.  More  productivity  and  more  transactions  per  time  and  em¬ 
ployee  are  achieved  without  increasing  work  strain  or  maybe  even  reducing  it.  Problem 
areas  can  be  tackled  that  were  never  suited  for  man  made  solutions  before.  Employees 
can  work  with  a  higher  degree  of  job  satisfaction.  They  will  be  able  to  look  upon  their 
work  as  good,  helpful,  efficient,  and  ethical. 

Modem  Technology  for  Information  and  Communicationf  Pros  and  Cons  of  the 
Ecological  Disaster 

Information  technology  is  also  frequently  sited  as  a  core  element  in  technical  environ¬ 
mental  protection.  Computers  in  conjunction  with  sensors  and  actuators  reduce  the  emis¬ 
sion  of  vehicles,  analyze  dioxin  concentrations,  reduce  power  consumption  of  freezers, 
localize  polluted  areas,  control  heating  systems,  and  detect  illegal  harvesting  of  the  rain 
forest.  Information  networks  help  prevent  environmental  pollution  and  the  abuse  of  valu¬ 
able  resources.  Telecommuting  delivers  an  important  ecological  contribution,  by  reduc¬ 
ing  the  resources  required  for  travelhng  and  driving  considerably,  especially  in  the  serv¬ 
ice  sector.  Construction  of  new  roads  will  be  unnecessary,  the  collapse  of  the  traffic  sys¬ 
tems  will  be  avoided.  The  people  will  not  need  to  move,  instead  the  data  would  travel. 
New  information  and  communication  techniques  present  an  image  that  is  ecologically 
acceptable:  no  malfunction,  no  air  pollution  and  no  exploitation  of  scarce  resources. 

But  the  ecological  impact  is  not  solely  positive.  Here  are  two  different  opinions:  the  first 
opinion  states  that  the  information  and  communication  technologies  give  us  hope  to  be 
delivered  from  the  problem  of  environmental  destruction  and  the  second  opinion  argues 
that  industrial  processes  and  western  ways  of  life  are  supported  and  reinforced  by  infor¬ 
mation  technology.  From  this  point  of  view  the  new  technologies  accelerate  the  con¬ 
sumption  of  ecological  resources  on  earth.  On  one  hand  the  networking  of  partners  using 
screens,  e-mails  and  modems  can  drastically  reduce  the  use  of  resources  and  nature.  On 
the  other  hand  the  development  of  these  techniques  from  an  ecological  point  of  view 
show  considerable  disadvantages.  The  ecological  balance  of  the  new  information  and 
communication  technologies  is  not  as  favorable  as  often  pretended.  According  to  calcu¬ 
lations  by  the  Institute  for  Climate  Research,  Environment  and  Energy  based  in  Wup¬ 
pertal,  Germany,  the  “ecological  package“  of  a  486  PC  adds  up  to  a  total  of  16  to  19  tons 
of  resources.  A  state  of  the  art  production  facility  for  semiconductor  wafers  consumes 
240.000  KWh  energy  per  day.  Additionally,  it  requires  6.8  million  liters  of  ionized  water 
and  70.000  cubic  meters  of  pure  nitrogen  per  day.  The  so-called  dematerialization  using 
electronic  processing  and  storage  proves  to  be  more  and  more  a  “green  mythos“.  Thus 
the  information  market  accelerates  the  waste  of  resources  to  a  great  extent.  This  can  be 
shown  even  more  drastically  in  connection  with  the  use  of  paper.  The  paper  consumption 
by  a  citizen  of  the  Federal  Republic  of  Germany  increased  from  126  kg  in  the  year  1970 
to  232  kg  now.  The  Frankfurt,  Germany  based  magazine  “Oko  Test“  reports,  that  there 
would  not  be  a  single  tree  on  earth  once  every  Chinese  person  had  the  same  paper  con¬ 
sumption  as  a  German,  There  are  many  other  unsolved  problems  of  ecology  with  respect 
to  information  and  communication  technology  such  as:  emission  of  electro-smog,  depo¬ 
sition  and  disposal  of  batteries,  chassis  and  printed  circuit  boards,  and  the  complete  dis¬ 
posal  of  worn  out  computers.  In  this  spirit  I  would  like  to  state  that  the  acceleration  of 


technical  and  sociological  developments  initiated  by  the  flow  of  information  and  com¬ 
munication  on  global  scale  contributes  directly  to  the  destruction  of  the  environment  and 
the  creeping  disintegration  of  society.  As  never  before  western  ideals  are  distributed 
around  the  globe  and  lead  to  demands  that  revolutionize  culture  and  way  of  life  of  other 
regions  at  a  breathtaking  speed.  At  the  same  time  we  recognize  an  increasing  polarization 
of  the  rich  and  poor,  especially  with  respect  to  information  access.  We  must  not  overlook 
that  today  about  50%  of  mankind  is  not  capable  of  using  a  telephone,  even  if  they  had 
access  to  one,  much  less,  the  capability  of  using  an  Internet  account.  A  new  ecology  of 
media  is  required  which  would  address  the  problem  of  disposal  of  the  inevitable  infor¬ 
mation  waste“. 

Effortless  Learning  or  New  Competition  for  Education 

To  receive  or  proliferate  information,  competence  must  be  developed  in  the  following 
areas:  the  ,Jaiow-how“  of  information  structure,  the  ability  to  evaluate  information  and  to 
make  it  usable,  the  development  of  searching  strategies,  and  the  use  of  communication 
according  to  established  rules.  Multimedia  development  offers  many  new  applications 
that  require  these  qualifications:  visualization  of  complex  structures  in  science,  new 
forms  of  work  such  as  tele  cooperation,  telediagnosis  in  medicine  and  technical  fields, 
telepublication,  teleleaming  and  -teaching. 

There  will  be  a  global  competition  in  the  once  national  educational  market.  It  is  therefore 
important  to  start  a  comprehensive  multimedia  initiative  for  education  and  network  ori¬ 
ented  teaching.  The  competitiveness  of  single  economies  can  only  keep  pace  with  global 
competition  if  network  style  thinking  in  school  education  and  university  teaching  is  es¬ 
tablished.  In  this  manner,  each  economical  location  wiU  be  assessed  as  a  scienctific  site 
and  as  a  place  for  information  and  communication.  There  will  be  a  great  new  demand 
everyone  to  successfully  export  sufficient  new  knowledge  into  the  global  competition. 
This  goal  cannot  be  achieved  by  the  industry  alone.  It  requires  a  new  and  strategic  coop¬ 
eration  between  industry  and  communities,  the  state,  and  private  or  social  educational 
institutions.  Multimedia  and  network  oriented  education  will  enrich  everyday  learning  in 
many  ways.  It  will  by  no  means  be  restricted  to  subjects  such  as  computer  science  as  it 
was  in  the  past.  More  than  ever,  new  media  must  be  accepted  and  established  as  a  tool  in 
all  subjects,  including  the  complete  course  of  all  subjects  beginning  with  the  scien¬ 
tific/technical  and  ending  with  the  musical/visual  arts.  The  currently  practiced  format  of 
multimedia  learning  using  e-mail  will  be  insufficient  in  the  future.  It  is  more  important  to 
acquire  expertise  in  the  practical  application  of  multimedia  communication  techniques, 
A  basic  requirement  for  this  is  the  education  of  teachers,  professors,  and  university  staff. 
Another  requirement  is  the  establishment  of  the  necessary  technical  equipment.  Taking 
into  account  all  of  the  investment  that  is  required  for  the  reorganization  of  education  us¬ 
ing  new  multimedia  learning  there  are  considerable  doubts  that  a  fast  implementation 
will  be  successful  using  public  or  community  money  alone.  The  universities  are  chal¬ 
lenged  at  the  same  time  to  reconsider  their  present  curricula  from  the  viewpoint  of  Mul¬ 
timedia  and  to  develop  pedagogic  didactic  concepts.  Focus  must  be  on  software  for  edu¬ 
cation  and  training.  The  development  of  “learning  on  demand“  must  be  a  subject  for 
further  research  in  order  to  understand  its  prerequisites  and  consequences  better.  One 
specific  special  danger  is  that,  on  different  levels,  at  school,  university,  or  administration, 
one  waits  for  the  other.  A  lot  of  time  will  be  lost,  as  previously  seen  in  many  European 
states.  Another  challenge  is  the  implementation  of  new  technologies  in  the  developing 


states,  which  requires  even  more  effort.  The  1996  initiative  in  Germany  for  “Internet  ac¬ 
cess  for  aU  schools“  must  be  further  developed  and  fully  implemented.  The  future  costs 
for  this  initiative  must  also  be  taken  into  account.  Along  these  lines,  it  should  be  pointed 
out  again  that  transformation  of  our  society  into  an  information-based  society  should 
happen  worldwide.  On  a  global  scope,  there  is  a  considerable  lag  in  the  developing 
countries,  especially  in  the  former  Soviet  Union  and  China.  We  clearly  must  see  that  in 
the  next  ten  years  more  new  knowledge  will  be  processed  and  taught  than  in  the  previous 
2500  years.  The  faster  and  the  more  comprehensively  we  address  this  task  the  greater  the 
chances  are  that  it  will  be  successfully  completed. 

Using  new  information  and  communication  technologies  must  become  a  general  cultural 
tradition,  including  media  education  in  professional  education.  Multimedia  and  network 
oriented  education  should  not  only  be  seen  as  only  technical  education  but  also  as  an 
opportunity  to  enrich  learning  playfully.  Using  these  new  technologies  naturally,  a  new 
didactic  and  content-oriented  creativity  will  be  possible  that  allows  more  room  for  per¬ 
sonal  activities  and  individual  learning.  As  the  teacher  acts  only  as  a  coach  to  help  ac¬ 
quire  knowledge,  independence  and  responsibility  of  the  student  become  more  and  more 
important.  This  proves  once  more  that  only  easy  access  to  Multimedia  will  help  achieve 
acceptance  of  new  technological  development  and  helps  promote  these  opportunities. 
Industry  is  also  responsible  for  promotion  of  media  education.  The  development  of  edu¬ 
cational  software  should  be  designed  for  international  use.  As  the  problem  of  establishing 
multimedia  techniques  in  industry  is  important  in  global  competition,  economics  must 
maintain  interest  in  encouraging  educational  initiatives  so  as  not  to  fall  behind  the  global 
standard. 

Emotions  and  Virtual  Reality 

The  ability  to  utilize  new  multimedia  opportunities  depends  solely  on  the  motivation  of 
the  end  user.  This  motivation  can  be  promoted  using  different  strategies:  the  opportunity 
can  be  presented  as  entertainment  or  it  can  facilitate  work.  Only  these  capabilities  will 
result  in  user  acceptance,  where  the  user  explores  each  new  interactive  possibility  step  by 
step.  Personal  motivation  is  -from  my  understanding-  the  key  around  which  all  of  us 
should  design  applications.  We  should  use  new  techniques  in  ways  that  make  them  emo¬ 
tionally  acceptable.  Emotional  relations  will  play  a  crucial  role  in  the  cyberspace  era. 
Several  studies  prove  that  cyberspace  will  change  emotional  relations  of  humans  at  work. 
In  addition,  new  emotions  will  be  created  to  replace  missing  real  contacts. 

Traditional  corporations  very  frequently  ignore  human  feelings  or  even  suppress  them. 
But  the  work  process  is  full  of  emotion.  Enthusiasm,  rage,  fear,  jealousy,  pride,  ambi¬ 
tion,  hatred,  decisiveness,  fury,  and  delight  are  all  possible  emotions  generated  by  the 
prospect  of  a  successfully  completed  project.  Negative  emotions  support  “work-to-rules“ 
tendencies,  positive  emotions  promote  willingness  to  perform,  productivity,  quality,  and 
finally  customer  satisfaction.  It  is  important  for  companies  to  motivate  their  employees. 
To  obtain  this  motivation,  stimuli  are  required  that  are  directly  connected  with  custom¬ 
ers’  wishes,  independent  of  the  size  of  the  office.  Whether  the  cyber  enterprise  possesses 
better  chances  to  develop  its  abilities  in  this  area  through  more  flexible  structures  has  yet 
to  be  shown.  We  must  remain  aware  that  the  customer  will  accept  not  everything  that  is 
technically  practicable. 


^7. 


The  integration  of  human  thinking,  acting,  and  decision  processes  is  one  of  the  greatest 
challenges  for  the  future  of  multimedia.  Neither  nature  nor  human  senses  utilize  digit- 
ized“  information.  Nearly  everything  is  “analog^.  It  is  not  important  to  man  that  human 
feelings  and  sensual  thrill  such  as  smelling,  tasting,  viewing,  listening,  and  loving  can  be 
implemented  digitally.  The  effort  of  scientists  to  understand  the  human  soul  as  an  infor¬ 
mation  processing  system  and  to  model  it  using  mathematical  equations  may  satisfy  their 
ambition,  but  is  of  little  help  in  eliminating  human  conflicts  and  contradictions  of  the 
presence.  Not  only  the  increasing  number  of  religious,  nationalistic,  and  racist  wars  sup¬ 
port  this  thesis,  but  also  the  inherent  conflicts  in  everyday  human  life.  The  new  ideology 
of  formalizing,  digitizing,  and  generaUy  making  everything  technically  controllable 
sweeps  the  world,  while  defining  power  under  the  cover  of  multimedia  democracy  and 
plurality.  Man  can  as  such  become  dependent  on  new  artificial/  technical  worlds  to  re¬ 
place  the  natural  environment,  forcing  him  to  rely  on  new  powers  such  as  Bill  Gates,  Leo 
B^rch,  Silvio  Berlusconi  and  others.  Along  this  line  critics  point  out  that  Multimedia  can 
lead  to  vicarious  satisfaction  and  virtual  reality  without  reference  to  the  real  world. 

The  resulting  consequences  of  Multimedia  on  mankind  and  the  possible  increase  of  cul¬ 
tural  differences  cannot  be  predicted  today.  Human  relations  through  electronic  channels 
will  serve  to  change  emotions  and  an  effect,  which  we  may  call  “vutualization“.  Prede¬ 
cessors  of  “virtual  children“  are  already  available  as  toys  and  for  those  who  do  not  want 
real  children.  A  proof  of  this  is  the  “Tamagochi“-cult  from  Japan.  Its  purpose  is  to  feed 
and  nurse  an  artificial  computer  egg  like  a  living  creature.  The  development  and  imple¬ 
mentation  of  “soul-machines“  as  described  in  the  previously  cited  article  “Emo  lebt“ 
(Spiegel  Spezial,  1996b)  documents  a  possible  trend.  This  trend  makes  me  very  thought¬ 
ful,  it  states  that  parents  sooner  or  later  may  be  satisfied  with  virtual  children  instead  of 
real  children.  The  blessings  of  Multimedia  are  Janus-faced.  Even  supporters  of  Multime¬ 
dia  must  concede  that  in  the  end  virtual  conference  dinners  have  no  taste. 

Change  in  Values  by  Multimedia 

The  transformation  of  our  society  into  an  information-based  society  requires  not  only 
state  or  private  regulations  worldwide,  but  also  self-obligation  and  responsibility.  The 
development  of  Multimedia  technology  will  essentially  change  political  systems  in  all 
countries.  This  has  already  been  seen  by  the  influence  of  media  on  the  breakdown  of 
communism.  Consequently  we  are  obliged  to  support  reforming  nations  and  not  con¬ 
struct  new  electronic  walls.  Many  new  ethic  and  moral  questions  must  be  investigated 
due  to  this  ongoing  virtualization. 

Because  of  conflict  caused  by  lost  values,  changes  of  values,  and  inflation  of  values,  per¬ 
sonal  attitudes  and  ways  of  thinking  gain  increasing  weight.  This  creates  great  new  chal¬ 
lenges  for  politicians  to  establish  regulations  reaching  much  further  than  traditional  laws 
which  take  morality  absolutely  seriously.  The  contradiction  on  all  levels  is  that  we  mean 
well  and  hardly  change  our  doings. 

Using  new  information  and  communication  technologies,  people  must  be  taught  new 
aims  for  their  life,  learning,  and  work.  Nobody’s  access  to  information  must  be  hindered 
by  another’s  monopoly  on  information.  In  conjunction  with  the  increasing  erosion  of 
states’  finances  in  all  countries,  a  collapse  would  be  inevitable  because  the  planned  de¬ 
velopments  could  not  be  paid  for.  In  this  scenario  new  workplaces  would  not  develop  and 


social  peace  could  not  exist.  The  unity  between  nations  would  not  be  promoted  but,  as 
presently  described  by  many  authors,  countries  would  be  further  differentiated  in  a  global 
competition.  In  this  case  the  technological  revolution  caused  by  Multimedia  and  data 
highways  could  become  “man-eating“  one  day  and  eliminate  their  own  children.  This 
could  happen  in  a  way  that  we  cannot  foresee,  as  we  could  not  foresee  the  catastrophes  of 
the  20^  century.  On  the  other  hand,  the  technological  revolution  could  also  produce  new 
values  promoting  living  or  working  together,  international  responsibility,  faimly  or  tradi¬ 
tional  values,  and  regional  affiliation  in  the  best  sense.  An  important  question  will  be 
whether  that  the  habits  of  man  will  change  as  quickly  as  media  technologies  and  how 
many  people  will  take  part  in  this  process,  A  pessimistic  view  on  the  development  of  the 
century  will  not  help  to  make  the  best  out  of  the  inevitable  evolution.  For  that  reason 
all  business  and  management  relevant  questions  should  be  viewed  with  critical  optimism. 
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Abstract 

Mobility  is  one  of  the  great  societal  demands  and  air  traffic  is  one  mode  of  transport  to 
serve  this  need.  Substantial  growth  rates  are  predicted  for  future  air  traffic  but  the  natu¬ 
ral  resources  of  real-estate,  airspace,  and  human  abilities  required  to  accommodate  much 
more  air  traffic  become  a  limiting  factor.  A  better  utilization  of  given  resources  is 
needed,  therefore,  to  increase  air  traffic  capacity.  One  attempt  in  this  direction  is  a  more 
efficient  air  traffic  management  by  the  introduction  of  more  automation  and  a  better  task 
sharing  between  the  automated  system  elements  and  the  human  operator.  A  reconsidera¬ 
tion  of  the  role  of  the  human  operator,  the  develop-ment  of  an  appropriate  automation 
strategy  and  appropriate  support  tools,  and  the  demonstration  of  concepts  for  a  future  air 
traffic  management  system  are  essential  steps  to  achieve  the  goal.  In  this  context  the 
particular  aspects  of  air  traffic  control  simulation  including  human  operators  for  the  test 
and  demonstration  of  air  traffic  control  concepts  are  discussed  and  recent  experiences  are 
presented. 


1.  Introduction 

Simulation  in  air  traffic  management  research  is  one  of  the  most  used  tools  to  achieve 
advances  in  air  traffic  management  in  Europe  and  world-wide.  Different  types  of  simu¬ 
lation  exist.  The  fast- time  simulation  of  air  traffic,  for  instance,  is  an  efficient  means  to 
assess  future  airspace  and  airport  capacities,  the  expected  flow  of  air  traffic,  and  the  de¬ 
lay  of  aircraft.  Other  topics  are  investigated  best,  however,  in  a  real-time  simulation  of 
the  respective  air  traffic  scenario.  These  topics  include  -  among  others  -  the  expected 
future  demands  on  air  traffic  controllers  and  pilots  or  the  performance  and  acceptability 
of  advanced  support  tools  onboard  an  aircraft  and  on  the  ground.  Other  areas  are  the  use 
of  a  future  air/ground  data  link  and  new  procedures  to  control  future  air  traffic  more  effi¬ 
ciently,  Real-time  simulation  is  the  right  choice  in  particular  if  human  operators  (air 
traffic  controllers,  pilots)  take  part  in  the  simulation. 


In  the  past  these  simulations  have  provided  administrations,  airport  authorities,  air  traffic 
control  authorities,  and  airlines  with  a  realistic  view  and  an  interactive  experience  of  fu¬ 
ture  air  traffic  management  tools  and  procedures.  Experiences  have  shown  that  the 
availability  of  a  high-performance  real-time  simulation  not  only  is  an  efficient  means  for 
the  design,  development,  assessment,  and  demonstration  of  future  air  traffic  management 
and  flight  guidance  systems  and  procedures.  It  is  also  a  prerequisite  for  a  successful  ac¬ 
quisition  of  attractive  new  research  contracts  in  air  traffic  management  regarding  auto¬ 
mation  and  the  future  role  of  the  human  operator  in  air  traffic  management. 


Air  traffic  control  (ATC)  is  that  element  of  air  traffic  management  (ATM)  which  exe¬ 
cutes  control  over  aircraft  en~route  and  in  the  terminal  area  of  airports.  In  the  following 
recent  experiences  in  ATC  simulation  with  particular  respect  to  the  human  operator  are 
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discussed. 


2.  Current  role  of  human  factors/engineering  in  ATC 

As  more  and  more  functions  of  the  human  operator  can  be  implemented  by  machines 
today  the  question  is  not  how  much  can  be  automated  but  what  degree  of  automation  is 
acceptable  responsibly.  This,  in  turn,  determines  the  role  of  the  human  operator  and  of 
its  interfaces  to  the  automated  system.  For  instance,  the  Concept  Document  for  the  fu¬ 
ture  European  Air  Traffic  Management  System  (EATMS)  lists  as  one  of  the  principles 
for  its  development  (EUROCONTROL,  1992): 

„The  concept  introduces  new  methods  of  task  sharing  between  the 
automated  system  elements  and  the  human  operator.  That  devel¬ 
opment  shall  be  carefully  balanced  in  particular  in  relation  to  the 
responsibilities  for  traffic  monitoring  and  decision  making;  this 
may  set  limits  to  the  degree  of  automation.  The  man/machine  in¬ 
terface  is  a  key  element  of  successful  application  of  automation 
and  of  overall  system  performance, “ 

The  forecast  of  the  potential  future  of  the  US  Air  Force  (USAF,  1995)  also  lists  among 
six  identified  essential  capabilities  „people“  which  includes  things  like  modeling  of  the 
human  operator,  training,  education,  and  human/machine  interaction. 

This  much  improved  role  of  „people“  and  associated  designs  and  developments  of  hu¬ 
man/machine  interactions  and  interfaces  in  comparison  to  the  situation  some  years  back 
cannot  be  overemphasized  in  times  of  severe  cuts  in  funding  and  rearrangements  of  de¬ 
velopment  priorities. 

3.  Some  fundamental  considerations 

Human  error  will  come,  regardless  of  the  quality  of  design  and  training.  Suppressing 
human  error  is  seen  as  a  naive  goal.  A  better  approach  would  be  to  correct  system  ergo¬ 
nomics  but  keep  the  human  in  the  loop  (Amalberti,  1997).  In  recognition  of  this  situa¬ 
tion  aviation  research  organizations,  air  traffic  services  and  airport  authorities  are  in¬ 
creasingly  interested  in  real-time  simulations  and  demonstrations  of  new  developments  in 
air  traffic  control  automation  before  any  final  decision  on  their  possible  introduction  is 
made.  Questions  of  expected  system  performance,  human  operator  workload  and  ac¬ 
ceptance  of  a  new  system  are  to  be  answered. 


Recent  reviews  and  studies  on  the  human  operator  and  automation  in  ATC  made  or  con¬ 
tracted  by  DLR  are  reported  by  Beyer  (1996).  According  to  the  findings  of  Lenorovitz, 
Olason,  Krois,  and  Tobey  (1991)  and  Jackson  (1989)  the  tasks  of  air  traffic  controllers 
can  be  summarized  as  follows: 


monitoring 

diagnosing 

planning 

deciding 

implementing 


what  is  the  state  of  air  traffic  ? 
what  happens  and  why  ? 
what  options  are  available  ? 
what  option  is  optimum  ? 
what  implements  an  option  best  ? 


A  major  issue  of  ATC  automation  is  the  transfer  of  cognitive  functions  from  the  human 
operator  to  machines.  This  approach  to  automation  in  ATC  may  lead  to  two  self- 
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contained  lanes  of  functionality:  one  is  present  in  the  human  operator  and  the  other  is 
implemented  by  the  machine.  Consequences  are: 

•  From  a  functional  and  procedural  point  of  view  the  machine  must 
be  able  to  handle  the  tactical  control  of  air  traffic  and  the  strategic 
management  of  the  flow  of  air  traffic  automatically. 

•  From  an  operational  point  of  view  there  would  be  a  partnership 
between  the  human  operator  and  the  machine:  The  human  operator 
decides  what  type  of  machine  support  suits  his  momentary  and 
foreseeable  needs  best  and  what  tasks  he  can  delegate  to  the  ma¬ 
chine  for  the  time  being. 

This  means  in  short:  Automate  as  much  as  technically  feasible  but  utilize  machine  func¬ 
tions  only  as  far  as  responsibly  acceptable.  By  this  approach  the  human  operator  will  be 
provided  with  a  suite  of  tools  to  select  from  according  to  his  needs  and  also  with  a 
chance  to  tune  the  tools  in  a  way  that  their  pace  is  matched  to  that  of  the  human  operator. 
Full  authority  is  assumed  by  the  human  operator  initially  when  new  functions  are  intro¬ 
duced  but  this  may  change  when  the  human  operator  gains  better  insight  and  more  con¬ 
fidence  into  the  new  machine  functions. 

4.  DLR^s  ATC  simulation  suite 

The  main  target  systems  for  air  traffic  management  research  and  development  are  air¬ 
ports,  aircraft,  the  airspace,  and  the  air  traffic.  For  obvious  reasons  it  is  next  to  impossi¬ 
ble  to  utilize  some  of  the  target  systems  directly  as  an  experimental  platform  to  investi¬ 
gate  new  concepts  of  human/machine  interaction  in  ATC  and  to  evaluate  new  products 
and  procedures.  In  1970,  therefore,  the  DLR  began  to  establish  virtual  worlds  of  air  traf¬ 
fic  control  and  aircraft  guidance  by  means  of  a  simulation  of  these  target  systems  with  a 
high  degree  of  realism.  Major  elements  of  the  simulation  are: 

Airspace  and  air  traffic  simulation 

A  data  base  provides  the  infrastructure  of  the  airspace  of  the  Federal  Republic  of  Ger¬ 
many  and  in  particular  of  the  extended  terminal  area  of  the  airport  of  Frankfurt/Main. 
The  data  base  includes  all  navigation  data  defined  by  ARINC  specification  424,  aircraft 
performance  and  weight  data  for  a  number  of  aircraft  types  as  well  as  several  different 
wind  profiles.  An  air  traffic  generator  uses  the  data  base  to  create  air  traffic  scenarios 
with  a  selectable  mix  of  aircraft  types  and  speeds,  routes,  wind  conditions,  departures, 
and  destinations.  These  scenarios  can  be  run  repetitively  to  investigate  the  influence  of 
new  air  traffic  structures,  operator  support  tools  and  operational  procedures  on  the  flow 
of  air  traffic,  the  resulting  air  transport  capacity,  and  the  workload  of  air  traffic  control¬ 
lers. 

Air  traffic  control  simulation 

The  DLR  Air  Traffic  Management  and  Operations  Simulator  (ATMOS)  employs  the  air¬ 
space  and  air  traffic  data  bases  and  the  air  traffic  scenarios  created  by  the  air  traffic  gen¬ 
erator  to  run  the  scenario  and  to  feed  the  radar  consoles  of  the  air  traffic  controllers  with 
realistically  looking  air  traffic  (fig.  1).  ATMOS  also  accommodates  new  support  tools 
for  air  traffic  controllers  to  be  investigated  in  a  simulated  operational  scenario.  A  group 
of  so-called  pseudo-pilots  (fig.  2)  monitors  the  simulated  aircraft,  communicates  with  the 
air  traffic  controllers  by  voice  or  data  link  and  implements  advisories  received  from  air 
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traffic  controllers  by  a  modification  of  the  state  of  aircraft  under  control.  The  air  traffic 
controllers  in  turn  monitor  the  aircraft  reaction  to  a  given  advisory  on  their  radar  screens 
thereby  closing  the  ATC  control  loop. 

Simulation  of  the  airport  of  Frankfurt/Main 

A  visual  simulation  of  the  airport  of  Frankfurt/Main  provides  views  as  seen  from  the 
working  position  of  the  tower  controllers  as  well  as  from  the  working  position  of  a  pilot 
approaching  at  /  departing  from  and  taxiing  on  the  airport  of  Frankfurt,  The  simulation 
includes  the  infrastructure  of  the  airport  (passenger  buildings,  hangars,  navigation  aids, 
runways,  taxiways,  etc.)  as  well  as  moving  objects  (approaching/  taxiing/departing  air¬ 
craft,  support  vehicles).  This  photo-realistic  representation  of  the  airport  is  visualized  by 
a  3-channel  backprojection  system.  The  three  screens  have  a  size  of  about  4x4  meters 
and  provide  a  view  of  about  150  degrees  horizontally  by  40  degrees  vertically. 

The  DLR  test  aircraft 

The  DLR  operates  the  jet  test  aircraft  ATT  AS  (Advanced  Technologies  Testing  Aircraft 
System)  with  a  rear  experimental  cockpit  and  a  bi-directional  air/ground  data  hnk.  The 
test  aircraft  carries  advanced  avionic  systems  which  co-operate  with  ATC  in  order  to 
develop  and  negotiate  an  optimum  space  and  time  based  flight  path  (4D  trajectory)  and 
to  exchange  situation  information,  sensor  data,  and  planning  information  between  the 
aircraft  and  the  ground. 

5.  Application  ofDLR's  ATC  simulation  suite 

The  views,  concepts,  and  tools  described  in  the  previous  chapters  form  the  background 
of  research  at  DLR  in  air  traffic  control  with  particular  emphasis  on  the  role  and  integra¬ 
tion  of  the  human  operator  in  a  future  ATC  system.  A  typical  application  is  presented 
next. 

First,  a  more  general  remark  on  ATC  simulations  in  a  European  context:  The  simulation 
of  a  future  European  ATC  system  lives  (and  sometimes  suffers)  from  the  fact  that  a  dif¬ 
ferent  education,  training,  and  cultural  background  of  air  traffic  controllers  must  be  con¬ 
sidered.  Air  traffic  controller  teams  participating  in  ATC  simulations  are  selected  by 
their  national  authorities  and  are  made  available  for  a  very  limited  period  of  time  only  at 
the  place  of  the  ATC  simulation.  No  homogeneous  nor  representative  group  of  test  sub¬ 
jects  can  be  expected.  An  attempt  is  made,  therefore,  to  reduce  differences  within  and 
between  the  groups  of  test  subjects  by  intensive  instruction,  training,  and  demonstration 
of  the  new  ATC  scenarios  and  support  tools  to  be  investigated.  This  requires  a  substan¬ 
tial  effort  and  consumes  a  major  part  of  the  time  budget  available  for  an  experimental 
investigation. 

5J  An  example:  the  PHARE  demonstration  PD/2 

A  successful  application  of  an  ATC  simulation  including  human  operators  was  run 
within  the  Programme  for  Harmonized  Air  Traffic  Management  Research  in 
EUROCONTROL  (PHARE).  The  objective  of  PHAM  is  to  organize,  co-ordinate  and 
conduct  studies  and  experiments  aimed  at  providing  and  demonstrating  the  feasibility 
and  merits  of  a  future  air-ground  integrated  air  traffic  management  system  in  all  phases 
of  flight.  The  results  of  the  program  should  help  to  refine  the  description  of  the  future 
Air  Traffic  System  concepts  needed  to  satisfy  demand  and  to  provide  information  on  the 
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best  transition  from  the  current  to  a  new  system  (EURCKTONTROL,  1997).  The  program 
is  currently  planned  to  extend  over  a  period  from  1989  to  1998  with  an  overall  cost  of 
about  91  MECU.  Participants  are  DERA  and  NATS  in  the  UK,  CENA  and  STNA  in 
France,  DFS  and  DLR  in  Germany,  RLD/LVB  and  NLR  in  The  Netherlands,  and 
EUROCONTROL.  DLR  was  in  charge  of  the  second  PHARE  Demonstration  (PD/2) 
which  focused  on  advanced  tools  for  the  management  and  control  of  arriving  air  traffic 
in  the  extended  terminal  area  of  an  airport. 

5.2  Experimental  set-up  of  PD/2 

The  tools  investigated  in  PD/2  were  designed  to  transfer  cognitive  and  planning  func¬ 
tions  of  air  traffic  controllers  to  machines  in  an  attempt  to  reduce  the  workload  in  a  fu¬ 
ture  air  traffic  scenario  with  much  higher  traffic  volume  than  today  (Schick  and  Tenoort, 
1997).  This  approach  to  more  efficient  man/machine  systems  in  ATC  addresses  the  fol¬ 
lowing  general  topics: 

•  time-based  tactical  and  strategic  control  of  air  traffic  in  the  ex¬ 
tended  terminal  area  of  an  airport, 

•  support  of  the  human  operator  by  „intelligent“  machines  in  con¬ 
trolling  arriving  air  traffic, 

•  delegation  of  tasks  to  machines  by  the  human  operator  while 
maintaining  ultimate  responsibility. 

An  important  assumption  was  that  in  the  future  more  and  more  aircraft  will  be  able  to 
calculate  and  to  implement  space  and  time  based  flight  trajectories  onboard  which  fulfill 
the  space  and  time  constraints  of  a  computer-based  manager  of  arriving  traffic  on  the 
ground.  Future  aircraft  will  be  able  to  negotiate  these  trajectories  with  the  arrival  man¬ 
ager  via  an  air/ground  data  link.  Aircraft  with  this  capability  are  called  4D-equipped 
aircraft.  In  PD/2  the  ATTAS  with  its  Experimental  Flight  Management  System  repre¬ 
sented  such  a  4D-equipped  aircraft. 

The  objectives  of  the  experiment  were: 

•  Determine  the  effect  of  various  automated  support  tools  on  air  traf¬ 
fic  controller  workload  and  arriving  air  traffic  flow. 

•  Determine  the  effect  of  4D-equipped  aircraft  on  air  traffic  con¬ 
troller  workload  and  arriving  air  traffic  flow. 

•  Determine  the  degree  of  air  traffic  controller  acceptance  of  new 
automated  support  tools. 

Four  test  organizations  were  set-up  as  follows: 

1.  A  reference  operating  mode  representing  current  tools  and  proce¬ 
dures  with  spatial  separation  of  arriving  aircraft  (OrgO). 

2.  An  advanced  operating  mode  with  automatic  space  and  time  (4D) 
based  planning  of  aircraft  trajectories,  detection/resolution  of 
planning  conflicts  on  the  ground,  and  a  novel  human/machine  in¬ 
terface  (Orgl), 

3.  Same  as  Orgl  but  with  30%  4D-equipped  aircraft  of  all  arriving 
aircraft  (Org2/30). 
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4.  Same  as  Org2/30  but  with  70%  4D-equipped  aircraft  of  all  arriv¬ 
ing  aircraft  (Org2/70), 

Each  of  the  organizations  was  run  with  a  medium  and  a  high  traffic  load.  The  term  ,4iie- 
dium“  represented  today’s  traffic  demand  while  „high“  corresponded  to  about  33%  more 
traffic  demand  than  ,^edium“.  Data  was  gathered  at  four  different  air  traffic  controller 
working  positions  (ACC  tactical,  Approach  Pickup/Planner/Feeder). 

Eight  air  traffic  controller  teams  each  consisting  of  4  controllers  from  7  European  nations 
(France,  Germany,  Italy,  The  Netherlands,  Romania,  Sweden,  United  Kingdom)  partici¬ 
pated  in  the  investigations. 

Eight  test  runs  per  team  -  balanced  by  order  -  were  recorded  and  analyzed.  These  in¬ 
cluded  runs  with  medium  and  high  traffic  load  for  each  of  the  four  organizations  OrgO  to 
C)rg2/70.  Quantitative  statistical  methods  and  non-parametric  statistics  based  on 
matched  pairs  of  observation  were  employed  for  data  analysis. 

5J  Data  acquisition  in  PD/2 

Data  was  gathered  in  the  following  categories: 

-  System  performance 

Number  of  landings,  flight  time,  inbound  delays,  precision  of  de¬ 
livery  and  flight  accuracy,  and  aircraft  separation. 

-  Air  traffic  controller  workload 

Objective  workload  indicators  included  the  number  of  ATC  in¬ 
structions  issued,  frequency  of  radio  communications,  and  per¬ 
centage  of  time  spent  for  radio  communication. 

The  subjective  workload  estimates  were  obtained  by  the  Subjective 
Workload  Assessment  Technique  (SWAT)  and  by  the  NASA  Task 
Load  Index  (TLX). 

-  Acceptance 

The  air  traffic  controller  acceptance  of  the  simulation  environment 
and  training,  human/machine  interfaces,  operational  procedures, 
tools  and  functions  was  assessed  by  post-run  debriefing  interviews 
and  questionnaires. 

In  addition  all  experimental  runs  were  video-taped. 

5,4  Representative  results  of  PD/2 
General  findings 

The  results  obtained  showed  that  all  teams  were  highly  qualified  and  motivated,  that  they 
accepted  their  role  in  the  experiment  to  test  a  simulated  and  not  a  pre-operational  ATC 
system,  and  that  they  passed  their  training  successfully  and  were  able  to  participate  in  all 
phases  of  the  experiment  without  problems.  The  teams  rated  the  set-up  of  the  experiment 
and  the  trials  as  being  realistic  and  valid. 

System  performance 

The  results  obtained  showed  that  a  medium  traffic  load  could  be  handled  almost  equally 
well  for  all  organizations  Qrg0-0rg2/70.  Under  high  traffic  load,  however,  it  was  imme¬ 
diately  evident  that  OrgO  reached  its  limits  and  that  air  traffic  was  handled  much  more 
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smoothly  with  the  more  advanced  organizations  Orgl-Orgl.  While  in  QrgO  about  62 
landings/hour  were  measured  this  value  increased  to  about  64  landings/hour  for  Orgl- 
Org2  for  the  same  traffic  demand  (for  comparison:  landing  one  B747  aircraft  more  per 
hour  (average)  at  Frankfurt/Main  in  reality  could  result  in  an  annual  profit  of  about  10 
Mio.  DM).  Furthermore  the  variability  of  the  number  of  landings  per  hour  as  well  as  of 
most  other  performance  parameters  like  flight  time,  inbound  delays,  precision  of  deliv¬ 
ery,  and  aircraft  separation  was  considerably  reduced  for  the  more  advanced  organiza¬ 
tions.  This  is  because  the  advanced  support  tools  provide  a  much  improved  predictabil¬ 
ity  of  the  expected  air  traffic  flow. 

Air  traffic  controller  workload 

The  objective  workload  indicators  showed  a  high  correlation  with  each  other  and  a  very 
uniform  pattern  of  results.  They  indicated  a  workload  reduction  or  no  significant  change 
of  workload  for  the  least  advanced  organization  Orgl  in  comparison  to  the  reference  or¬ 
ganization  OrgO  for  the  air  traffic  controller  working  position  with  the  highest  load 
(Pickup).  However,  for  the  more  advanced  organizations  Org2/30  and  Org2/70  a  signifi¬ 
cant  reduction  of  workload  was  measured  for  all  air  traffic  controller  working  positions. 

The  subjective  workload  estimates  collected  by  means  of  SWAT  and  TLX  also  correlated 
well  with  each  other.  The  Pickup  controllers  rated  their  own  workload  considerably 
lower  when  they  worked  under  Orgl  in  comparison  to  OrgO.  The  Feeder  controllers  did 
not  indicate  any  difference  whereas  the  ACC  West  controllers  experienced  a  bit  more 
workload  when  changing  from  OrgO  to  Orgl.  However,  the  more  advanced  organiza¬ 
tions  Org2/30  and  Org2/70  led  to  a  reduction  of  perceived  workload  for  all  controller 
working  positions. 

The  objective  as  well  as  the  subjective  measures  of  workload  showed  the  same  trend  in 
workload  reduction  for  the  more  advanced  organizations  together  with  some  re¬ 
distribution  of  workload  between  the  Approach  and  ACC  working  positions.  But  be¬ 
cause  of  the  low  overall  workload  level  of  the  ACC  working  position  the  slightly  greater 
workload  for  Orgl  and  Org2  was  considered  to  be  within  acceptable  limits. 

Acceptance 

The  human/machine  interface  characteristics  (mouse  as  input  device,  availability  of  pop¬ 
up  menus,  display  design  such  as  color  coding,  display  options  like  highlighting  a  se¬ 
lected  trajectory)  received  a  high  degree  of  controller  approval.  Most  operational  aspects 
(support  for  safe  traffic  handling  and  „maintaining  the  picture“,  concept  of  handling  4D- 
equipped  and  non-equipped  aircraft,  partitioning  of  tasks  between  working  positions) 
were  also  very  well  accepted. 

From  the  results  obtained  by  this  experiment  it  was  concluded  that  the  concept  of  the 
more  advanced  organizations  to  control  arriving  traffic  at  an  airport  has  proven  its  suit¬ 
ability.  The  concept  has  shown  some  potential  to  improve  air  traffic  throughput  and 
quality  of  service.  Improvements  can  be  achieved  at  acceptable  levels  of  air  traffic  con¬ 
troller  workload.  Although  the  concept  was  approved  by  the  participating  air  traffic 
controllers  there  is  room  for  further  improvements  of  the  automated  support  tools. 
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6.  Further  human/machine  interface  design  experiences  and  observations 
The  PHARE  demonstration  was  only  one  of  a  number  of  experimental  investiga¬ 
tions  in  ATC  at  DLR  with  particular  respect  to  the  human  operator.  Further  ob¬ 
servations  in  similar  experimental  investigations  showed  that: 

•  Concerns  were  raised  that  better  automation  tools  designed  to  enhance 
the  performance  of  air  traffic  controllers  may  take  away  interesting 
tasks  from  the  human  operator  he/she  is  able  and  willing  to  perform. 

The  new  tools  may  render  certain  jobs  redundant. 

•  New  monitoring  tools  provided  information  normally  not  available  to 
the  air  traffic  controller  which  was  considered  sensitive  by  the  ATC 
authority.  Selective  access  mechanisms  had  to  be  designed  into  the 
tools  to  control  the  availability  of  information  to  different  parties. 

•  A  new  monitoring  tool  documented  encounters  of  the  legal  operating 
limits  too  close  sometimes  when  air  traffic  controllers  tried  their  best 
to  resolve  a  demanding  situation.  This  type  of  documentation  was  not 
available  in  the  past.  Adverse  effects  on  human  operator  motivation 
and  performance  were  expected. 

•  When  air  traffic  controllers  were  provided  with  the  opportunity  to  se¬ 
lect  alternative  colors/color  schemes  for  a  graphical  user  interface, 
only  a  small  amount  of  experimentation  with  colors  was  observed. 

The  air  traffic  controllers  stayed  with  the  color  scheme  offered  by  the 
design. 

•  In  one  case  the  color  convergence  and  the  focus  of  the  monitors  were 
not  optimum.  Therefore  a  light  blue  background  of  the  screens  was 
chosen  to  avoid  too  many  different  non-convergent  colors  and  to 
achieve  a  certain  level  of  apparent  sharpness  of  the  image.  Knowing 
the  background  of  this  decision  the  air  traffic  controllers  accepted  the 
design  very  well. 

•  A  color  scheme  for  ATC  consoles  developed  with  great  effort  and  care 
by  one  European  nation  and  accepted  by  their  national  air  traffic  con¬ 
trollers  did  not  find  the  same  degree  of  acceptance  by  air  traffic  con¬ 
trollers  of  other  European  nations. 

These  and  similar  observations  and  experiences  are  going  to  be  incorporated  into  human 
factors  design  guides  for  ATC  systems.  An  example  of  such  guide  is  the  documentation 
„Human  Factors  in  the  Design  and  Evaluation  of  Air  Traffic  Control  Systems^  by  Car- 
dosi  and  Murphy  (1995). 

7,  Conclusions 

Despite  the  progress  of  automation  in  air  traffic  management  and  control,  the  role  of  the 
human  operator  in  arriving  at  safe,  economic,  and  acceptable  ATC  systems  of  high  per¬ 
formance  gains  increasing  importance.  This  is  accompanied  by  a  growing  demand  of 
real-time  ATC  simulations  to  test  and  to  demonstrate  the  benefits  and  acceptance  of  new 
automated  support  tools  with  particular  respect  to  the  role  and  the  work  of  air  traffic 
controllers  before  a  final  decision  about  their  possible  introduction  into  operational  use  is 
made.  Experience  has  shown  that  human  factors/engineering  knowledge,  an  advanced 
automation  strategy  and  a  real-time  ATC  simulation  in  combination  are  a  sound  basis  to 
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Figure  1:  DLR  Air  Traffic  Management  and  Operations  Simulator  (ATMOS). 


Figure  2:  ATMOS  pseudo-pilot  working  positions. 
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Abstract 

Simulation  is  the  traditional  method  for  the  development  of  Air  Traffic  Control  (ATC) 
systems.  The  Eurocontrol  Experimental  Centre  houses  an  extremely  flexible  and 
advanced  ‘Real-Time’  simulator,  'which  is  employed  for  the  operational  evaluation  of 
developments  in  specific  ATC  systems,  and  for  the  development  of  generic  future 
systems  (ODID,  PHARE  etc.).  Real-time  simulations  are  expensive  (about  100,000  ECU 
per  hour),  require  skilled  staff  and  participant  controllers  (up  to  50  controllers  and  20 
staff)  and  take  about  a  year  to  prepare.  Faster  and  cheaper  alternatives  have  therefore 
been  developed.  ‘Fast-time’  simulation  uses  a  computer-based  simulation,  where  the 
human  controller  is  replaced  by  pre- defined  algorithms  and  data  values,  and  exists  at 
various  levels  of  detail  (RAMS,  EAM,  SIMMOD).  A  single-person  Real-Time 
simulator,  (TRACON/Pro)  with  Al-based  synthetic  ‘pilots’  and  speech  generation  and 
recognition  is  used  for  the  evaluation  of  measuring  tools.  ‘Rapid  Prototyping’  is  used  for 
the  development  and  initial  investigation  of  new  displays,  ‘Part- task’  simulations  which 
investigate  specific  aspects  of  human  performance  are  used  to  provide  data  on  the  speed 
and  accuracy  of  human  operators  in  ATC-related  tasks.  These  data  are  used  in  Fast-Time 
simulation,  and  in  operator- oriented  simulations  using  generic  and  special-purpose 
Micro-SAINT  models,  to  investigate  generic  changes  to  existing  methods  -  use  of 
computer-based  inter-sector  transfers,  or  multi-sector  planning.  Finally,  small  simplified 
models  are  developed  on  PC-compatibles  to  demonstrate  more  radical  approaches  to 
ATC.  (DEMON  and  DEMFAST,  for  example,  show  the  effects  of  a  radical  re-structuring 
of  the  ATC  system  using  data-links,  computer-based  conflict  monitoring  and  pre¬ 
evaluation  of  conflict  resolution  strategies). 

Introduction 

Eurocontrol  is  the  European  Organisation  for  the  Security  of  Air  Navigation.  It  is  an 
international  organisation  established  by  treaty,  to  which  24  nations  are  currently 
signatories.  The  purpose  of  the  organisation  is  to  improve  Air  Traffic  Control  procedures 
throughout  Europe. 

The  Eurocontrol  Experimental  Centre,  at  Bretigny-sur-Orge,  near  Paris,  is  concerned 
with  the  evaluation  and  development  of  ATC  systems,  and,  to  a  lesser  extent,  with 
fundamental  research  into  Air  Traffic  Control.  Simulation  has  traditionally  been,  and 
remains,  the  major  tool  for  these  activities.  The  ways  in  which  simulation  is  employed 
have  diversified  in  recent  years,  so  that  the  Centre  now  provides  an  instructive  showcase 
for  simulation  methods  for  the  investigation  of  complex  multi-agent  dynamic  control 
systems. 

“Horses  for  courses”  is  a  traditional  English  proverb,  meaning,  literally,  that  you  should 
choose  a  horse  which  suits  the  type  of  race  you  are  riding  in.  In  general,  you  should 
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choose  a  tool  that  fits  the  task  you  are  undertaking.  In  particular,  in  ATC  research,  you 
should  choose  the  type  of  simulation  method  to  suit  the  question  you  are  asking. 

Real-Time  Simulation 

The  Eurocontrol  Experimental  Centre  was  originaUy  devoted  to,  and  literally  built 
around,  a  Real-Time  simulator.  When  this  was  originally  built  in  1967,  it  was  the  first 
digital  Real-Time  simulator  in  the  world,  based  on  a  Telefunken  TR4  digital  computer.  It 
was  then,  and  probably  remains,  the  only  large  simulator  which  is  not  tied  to  a  particular 
real  system.  Each  of  the  main  components  of  the  simulator:  computer;  piloting 
equipment;  control  room;  video  and  recording  systems  has  been  replaced  several  times  in 
the  last  thirty  years,  but,  like  Jack  Hobbs’s  cricket  bat,  after  three  new  handles  and  five 
new  blades  it  is  just  as  good  as  new.  The  current  simulator  is  capable  of  running  two 
large-scale  simulations  simultaneously.  Simulated  control  rooms  are  constructed  for 
each  simulation,  corresponding  to  the  physical  layout  of  the  existing  or  proposed  future 
system.  Up  to  40  control  positions  can  be  simulated  simultaneously,  representing  up  to  a 
dozen  sectors  in  one  or  more  ATC  centres.  These  control  positions  resemble  the  existing 
positions,  providing  similar  functions  to  the  controller.  Strip  bays  and  strip  printers  can 
be  provided  where  needed,  although  Electronic  Data  Displays  are  increasingly  used. 
Monochrome  or  colour-coded  radar  displays  can  be  provided,  using  various  sizes  of 
display  up  to  20-inch  (50  cm  )  square  raster-scan  displays  showing  2,000  by  2000  pixel 
images  in  full  colour.  Additional  Teed’  sectors  are  manned  by  EEC  Controllers,  who 
simulate  the  adjacent  and  underlying  sectors.  One  or  more  ‘pilot-operators  simulate  the 
pilots  of  the  aircraft  in  each  sector.  These  pilot-operators,  some  of  whom  are  qualified 
pilots  in  reality,  translate  the  controller’s  orders  into  instructions  to  the  computer,  and 
speak  the  computer-generated  messages  (position  reports,  etc.)  to  the  controllers.  Several 
hundred  aircraft  may  be  simulated  simultaneously.  Simulated  RTF  and  ground 
communications  are  provided  and  can  be  recorded.  Where  needed,  data-links  can  be 
simulated.  Elaborate  video-recording  equipment  is  available,  and  a  special  system,  Ergo- 
ISA  (Instantaneous  Stress  Analysis)  (Hering  and  Coatleven,  1996)  can  be  used  to  provide 
a  virtually  ‘on-line’  self-assessment  of  controller  strain  from  each  working  position, 
displayed  at  the  central  simulation  control  position. 

By  any  standards,  this  is  a  large  system.  The  major  costs  lie,  not  in  the  hardware,  but  in 
the  software  of  the  system.  Each  Real-Time  simulation  consists  of  ten  to  forty  exercises, 
each  lasting  one  to  two  hours,  spread  over  three  to  six  weeks.  For  each  simulation,  route 
structures,  sector  layouts  and  traffic  must  be  defined.  Specific  software  functions  must  be 
programmed  and  communications  and  other  links  prepared  and  checked,  (Our  previous 
chief  programmer,  retiring  after  25  years’  experience,  told  me  that  he  had  never  received 
a  specification  which  did  not  have  at  least  one  major  omission,  sufficient  to  make  the 
system  inoperable.)  The  programming  of  a  large,  distributed,  Real-Time  simulation  of 
this  sort  is  very  specialised  skill,  and  is  only  slightly  less  complex  than  programming  a 
real  system.  Even  when  the  software  is  as  close  to  complete  as  can  be  managed,  the  data 
representing  the  traffic  must  be  prepared  with  extreme  care,  since  controllers  are  very 
sensitive  to  abnormal  traffic,  such  as  heavy  aircraft  programmed  to  land  on  grass 
airfields.  Apparently  trivial  inconsistencies  can  greatly  affect  the  willingness  of 
controllers  to  accept  the  realism  of  the  simulation.  About  fifteen  skilled  staff  are  involved 
in  the  programming  of  current  simulations  and  anticipating  future  requirements.  A 
further  ten  are  employed  in  collecting  and  preparing  traffic  data.  (This  is  never  entirely 
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routine,  even  when  a  simulation  is  concerned  with  current  and  near-future  traffic.  When 
‘future  systems’  are  being  simulated,  it  is  particularly  difficult.)  Each  simulation  is 
controlled  by  an  experienced  project  leader,  who  is  an  experienced  controller,  with  an 
assistant.  Although  the  time-scale  for  a  Real-Time  simulation  is  about  twelve  to  eighteen 
months,  and  we  normally  run  about  ten  simulations  a  year,  most  project  leaders  are 
involved  in  several  simulations  at  the  same  time, preparing  one  simulation,  managing  a 
second  and  reporting  another.  If  we  include  the  staff  involved  in  procuring  and 
maintaining  equipment,  in  analysing  the  50  megabytes  of  data  from  a  routine  simulation, 
producing  training  and  briefing  programs  and  documents  and  interim  and  final  reports, 
finding  accomodation  for  visiting  controllers  and  for  management  visitors  during  the 
simulation,  about  a  hundred  staff  are  involved  in  the  running  of  the  Real-Time  simulator. 

A  Real-Time  simulator  exercise  is  very  expensive.  If  a  technician  fails  to  switch  on  the 
recording  system  for  a  ninety-minute  exercise,  it  costs  the  equivalent  of  totally  destroying 
the  Centre’s  two  best  offical  cars. 

A  Real-Time  simulation  is  not  undertaken  lightly.  The  yearly  programs  are  negotiated 
several  years  in  advance.  It  takes  about  six  to  nine  months  to  prepare  a  simulation,  one  or 
two  months  to  run  it  and  three  or  four  to  analyse  it,  although  the  major  results  are 
usually  obvious  during  the  running,  and  the  customer  often  puts  them  into  application 
without  waiting  for  the  formal  report. 

Real-time  simulation  is  best  used  as  a  final  verification  of  a  well-researched  system,  or 
for  revisions  of  the  airspace  of  an  existing  system.  It  is,  however,  also  used  to  investigate 
potential  future  systems. 

Two  major  series  of  simulations  have  been  carried  out  at  EEC  to  investigate  some 
significantly  different  systems.  The  first  series  of  simulations  (ODID  -  Operational 
Displays  and  Input  Devices)  took  place  in  1987  (ODID  I,  Prosser  and  David,  1988a), 
1988  (ODID  n,Prosser  and  David,1988b),  1990-1991  (ODID  m,Prosser  et  al  1991)  and 
1993  (ODID  IV,Graham  et  al,  1991).  These  studied  ways  of  presentation  increasingly 
remote  from  the  traditional.  ODID  IV,  in  fact,  employed  a  widowing  system  using 
several  innovative  forms  of  display,  and  radar  labels  that  could  be  expanded  to  provide 
full  flight  plans,  and  allowed  the  controller  to  record  his  instructions  to  the  aircraft 
directly  through  the  label.  A  great  deal  of  attention  was  paid  to  the  detailed  ergonomic 
design  of  the  display,  which  led  to  general  recommendations  (Jackson  and  Pichancourt, 
1995),  the  use  of  colour  being  based,  for  example,  on  the  recommendations  of  Reynolds 
and  Metcalfe  (1992). 

The  second  set  of  simulations  (PHARE  -  Program  for  Harmonised  ATC  Research  in 
Europe  )  began  with  PD/1  (Autumn  1995),  has  now  reached  PD/2  (Autumn  1996),  and 
will  culminate  in  PD/3  (Spring  1998).  These  simulations  involve  radical  innovations, 
such  as  the  linking  of  Real-Time  simulators  in  different  member  states,  and  data-link 
transmissions  to  real  aircraft  in  flight.  The  financing,  organisation  and  administration  of 
such  simulations  involve  tremendous  efforts,  and  take  a  very  long  time. 

From  a  research  point-of-view,  Real-Time  simulation  has  other  drawbacks.  It  is  rarely 
possible  to  repeat  an  exercise,  since  the  human  variations  introduced  by  the  feed  sector 
controllers  and  pilots  introduce  variations  in  the  exact  sequences  of  traffic,  leading 
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controllers  to  adopt  different  solutions  to  problems,  so  that  the  repeat  exercise  diverges 
from  the  original.  Even  where  a  close  repetition  is  possible,  the  controller,  being  human, 
remembers  his  previous  actions,  particularly  if  he  made  mistakes,  and  does  better  the 
second  time  round.  Comparisons  between  controllers  are  difficult,  since  each  controller 
has  a  repertoire  of  preferred  tactics,  based  on  his  working  experience.  Controllers  tend  to 
judge  a  system  as  a  whole,  so  that  a  technically  minor  flaw  may  lead  them  to  reject  an 
otherwise  promising  system  out  of  hand.  On  the  other  hand  they  often  display  a  can-do 
attitude,  finding  ways  around  flaws  in  official  procedures,  and  sharing  duties  within  a 
sector  so  that  expected  overloads  do  not  happen.  Admirable  as  this  trait  may  be  in  real- 
life,  it  can  totally  distort  a  carefully  planned  simulation.  Finally,  however  much  effort  is 
devoted  to  training,  controllers  continue  learning  throughout  a  simulation.  (In  real  life,  it 
takes  a  controller  up  to  six  months  to  ieam’  a  new  sector  -  in  simulation  he  rarely 
occupies  the  same  position  for  as  much  as  a  week).  Properly  balanced  experimental 
designs  are  extremely  hard  to  achieve. 

Fast-Time  Simulation 

Historically,  the  first  alternative  to  Real-Time  simulation  was  Fast-Time  simulation.  This 
technique  is  purely  computer-based,  and  uses  essentially  the  same  data  generation  and 
analysis  procedures,  but  replaces  the  actual  simulation  by  a  computer-based  simulation. 
While  the  software  of  the  Real-Time  simulator  calculates  and  presents  successive  images 
with  a  fixed  cycle  time  (usually  5  or  ten  seconds),  a  Fast-Time  simulator  maintains  a 
store  of  future  events,  and  carries  then  out  in  succession  as  fast  as  it  can,  advancing  its 
‘internal  clock’  accordingly.  If  there  is  not  much  traffic,  or  if  the  level  of  detail  required 
is  low,  this  process  may  run  much  faster  than  Real-Time.  On  other  occasions,  ‘Fast- 
Time’  may  run  slower  than  ‘Real-Time’.  The  original  Eurocontrol  Fast-Time  model 
(EAM)  has  been  through  almost  as  many  modifications  and  revisions  as  the  Real-Time 
simulator,  and  its  current  avatar,  called  RAMS  (Revised  Arithmetic  Simulator),  is 
designed  to  share  data  generation  and  analysis  software  with  the  Real-Time  system. 
Since  human  intervention  is  essentially  eliminated,  repeated  runs  of  the  same  exercise 
should  give  identical  results.  Estimates  of  the  probable  workloads  on  controllers  can  be 
obtained,  using  a  post-processor,  which  allocated  pre-defined  quantities  of  effort  or 
working  time  for  specific  events.  Fast-time  simulation  is  inherently  cheaper  and  more 
consistent  than  Real-Time  simulation,  but  it  lacks  the  error- detecting  facility  and  self- 
awareness  of  Real-Time  simulation.  Where  a  controller  in  Real-Time  simulation  will 
stream  aircraft  so  that  potential  conflicts  do  not  happen,  the  Fast- time  simulator  will 
‘solve’  individual  conflicts  as  they  occur,  with  no  strategic  insight. 

Practically,  Fast-Time  simulations  take  nearly  as  long  as  Real-Time  simulations,  since 
the  preparation  and  analysis  phases  are  essentially  similar.  More  exercises  can  be  carried 
out,  since  a  few  hours,  or  days,  of  computer  time  are  now  cheap,  and  simple  methods  can 
be  developed  for  modifying  traffic  samples  to  generate  minor  variation.  Fast-Time 
simulation  may  be  used  by  itself,  or  as  a  preliminary  to  a  Real-Time  simulation  to  weed 
out  the  more  unpromising  alternatives.  (Vergne  and  Tewes,1996) 

A  Fast-Time  simulation  requires  two  or  three  months  of  preparation,  about  a  month  of 
running  and  about  a  month  of  analysis  and  reporting.  About  ten  people  are  involved, 
mainly  in  data  preparation,  in  liaison  with  the  client  and  in  analysis.  Controllers  are  not 
required  as  such,  although  the  team  must  contain,  and  is  usually  led  by,  staff  with 
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considerable  control  experience  who  can  communicate  effectively  with  the  client. 

SIMMOD 

SIMMOD,  developed  by  the  US  Federal  Aviation  Authority,  is  similar  to  the  Eurocontrol 
Fast- Time  simulator,  except  that  it  is  more  oriented  towards  airports  and  that  it  has  some 
technical  constraints  on  traffic  behaviour.  SIMMOD  can  be  used  to  study  the  ground 
movement  of  aircraft,  for  which  the  EAM  and  RAMS  are  not  suitable.  It  is  not  concerned 
with  individual  working  positions,  but  with  presenting  a  global  appreciation  of  a  set  of 
alternative  situations. 

SIMMOD  studies  are  mainly  concerned  with  the  physical  movement  of  aircraft,  and  the 
alterations  to  be  expected  if  procedures  or  facilities  are  changed.  Watkins  et  al  (1997) 
investigated  the  reduction  of  separation  that  could  be  applied  if  a  moderate  cross-wind 
was  blowing  wake  vortices  away  from  the  runway,  for  example. 

In  general,  SIMMOD  studies  take  two  or  three  months,  and  involve  two  or  three  staff. 

TRACON 

The  TRACON/Pro  simulator  is  a  single-position  autonomous  Real-Time  simulator. 
Initially  designed  as  a  training  simulator  in  the  USA,  we  use  it  as  a  test  bed  for 
measurement  techniques  for  the  main  Real-Time  simulator  and  for  reality.  It  has  some 
unusual  features.  Traffic  can  be  generated  relatively  easily  to  produce  representative 
samples,  and  can  be  made  to  behave  in  a  more  realistic  manner  than  is  usually  the  case  in 
our  main  Real-Time  simulator.  Pilots  can  be  ‘programmed’  to  miss  or  misunderstand 
orders,  and  will  refuse  impossible  or  incorrect  manoeuvres.  Weather  problems  and 
communications  failures  can  be  simulated  if  desired.  It  can  operate  using  voice 
recognition,  provided  the  controller  observes  strict  ICAO  phraseology,  and  provided  that 
he  trains  the  system  to  recognise  his  voice.  This  task  takes  one  to  two  hours,  and  leads  to 
an  average  recognition  rate  of  94%  allowing  three  repeats,  in  active  control.  The  system 
generates  a  ‘score’  for  each  run,  awarding  points  for  correctly  controlled  aircraft,  and 
deducting  them  for  errors,  and  to  a  lesser  extent  for  redundant  orders.  A  first  study 
(David  and  Pledger,  1995)  showed  that  control  using  speech  recognition  was  more 
efficient  than  control  using  keybord  input  of  code  instructions,  possibly  because  the 
controller  did  not  need  to  divert  his  attention  from  the  screen  whem  using  voice  control. 
The  same  study  showed  that  both  the  SWAT  (Subjective  Workload  Assessment 
Technique)  (Reid  and  Nygren  1988)  and  ISA  (see  above)  produced  a  measurable 
decrease  in  control  efficiency.  We  are  currently  using  the  TRACON/Pro  system  to 
evaluate  some  potential  measures  of  ‘strain’,  incuding  the  analysis  of  cortisol  in  saliva, 
and  a  test  of  the  relative  incidence  of  alpha-rhythm  in  the  EEG  with  eyes  open  or  shut. 
We  intend  to  use  it  in  future  to  evaluate  other  methods  and  to  obtain  base-line  data  on  the 
time  taken  to  perform  common  ATC  operations. 

TRACON/Pro  comes  as  a  unit,  including  compiled  software.  We  are  not,  therefore,  able 
to  modify  the  human-computer  interface  in  line  with  modem  developments,  but,  as  a 
basic  test-bed  it  is  sufficiently  close  to  current  practice  as  to  be  accepted  by  controllers 

(A  simple  version  of  TRACON/Pro,  lacking  a  separate  screen  for  strips,  strip  printer  and 
voice  recognition  facilities,  is  available  as  a  computer  game  under  the  name  TRACON  H. 
We  recommend  this  to  students  or  experimenters  needing  a  basic  ATC  simulation.) 
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A  TRACON/Pro  study  takes  about  six  weeks  of  planning,  one  to  three  days  of  controller 
participation  for  each  controller,  and  a  month  of  analysis. 

Rapid  Prototyping 

Rapid  Prototyping  can  in  principle  be  used  for  the  development  and  initial  investigation 
of  new  and  unorthodox  ATC  displays  and  methods  (Broadbent  1993).  Although  sonie 
studies  can  be  done  on  static  images,  it  is  generally  necessary  to  provide  dynamic 
images,  which  implies  a  background  ATC  model.  Since  communications  are  an  integral 
part  of  ATC,  it  becomes  necessary  to  provide  some  form  of  ‘pilot-operator’  and  some 
form  of  ‘feed  controller’.  Traffic  must  be  more-or-less  realistic,  so  that  the  rapid 
prototyping  system  becomes  a  miniature  Real-Time  simulator,  with  all  the  administrative 
and  organisational  problems  that  implies.  For  the  moment,  we  are  not  pursuing  this 
approach,  although  it  may  be  necessary  to  meet  some  future  problems. 

GENERIC  Micros AINT  Model 

The  ‘GENERIC’  model  is  a  Fast-Time  simulation  model  written  in  MicroSAINT,  which 
is  an  event-based  discrete-event  simulation  system.  Micro-SAINT  provides  the 
underlying  operating  and  analysis  facilities  and  permits  the  user  to  define  linked  chains 
of  events.  GENERIC  is  essentially  a  mirror-image  of  RAMS  and  SIMMOD.  Aircraft  are 
defined  only  by  their  times  of  flight  through  successive  sectors.  Each  sector  is 
represented  by  a  network  which  itself  contains  networks  representing  the  activities  of 
procedural  and  executive  controllers.  The  tasks  within  the  networks  representing  the 
individuals  may  require  to  joint  activities  of  one  or  more  individuals,  and  reqire  times 
that  may  be  fixed,  normally  or  exponentially  distributed.  The  probabilities  that  additional 
tasks  will  be  required  are  defined  by  expressions  reflecting  the  traffic  density.  Times 
befom  which  tasks  must  be  performed  are  carefully  noted,  and  sequential  dependencies 
observed.  (For  example,  the  executive  controller  cannot  alter  the  flight  path  of  an  aircraft 
before  the  procedural  controller  has  treated  it. )  The  number  of  occasions  where  a  task  is 
not  completed  in  time  is  recorded,  as  are  the  total  occupation  times  for  each  controller. 

We  have  used  this  model  to  investigate  the  probable  workloads  resulting  from  grouping 
several  executive  controllers  under  one  procedural  controller,  and  the  effects  of 
introducing  a  strategic  ‘super-planner*  (David,  1997b) 

Using  this  model  requires  a  sound  grasp  of  the  principles  of  MicroSAINT,  and  close 
attention  to  the  definitions  of  priority  of  tasks.  It  also  requires  reliable  figures  for  the 
duration  of  standard  ATC  tasks.  Subjective  estimates,  widely  used  for  this  purpose,  have 
been  shown  to  be  unreliable,  and  the  few  available  ‘objective’  measurements  are,  to  put  it 
mildly,  obsolete  (David  1972). 

We  hope  to  obtain  more  reliable  figures  by  analysis  of  video  and  audio  recordings  of 
TRACON  exercises,  and  to  adapt  the  GENERIC  model  to  a  more  modem  version  of 
MicroSAINT. 

We  will  then  use  the  model  to  complement  Fast-time  and  other  studies  of  aircraft 
movement  to  confirm  that  the  corresponding  workloads  for  controllers  can  be  carried  out 
in  the  available  time,  to  make  more  detailed  examinations  of  the  consequences  of 
assumptions  about  workload  distributions,  (For  example,  assuming  that  all  activities  take 
exactly  the  average  time  produces  generally  optimistic  figures,  while  the  times  taken  for 
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most  human  ATC  activities  tend  to  be  at  least  exponentially  distributed)  and  to  examine 
the  consequences  of  changing  operating  methods.  (For  example,  verbal  coordination 
requires  the  simultaneous  attention  of  both  parties  -  computer-moderated  coordination 
requires  only  sequential  attention.) 

Part-Task  Simulations 

Part-task  simulations  can  be  used  to  study  specific  aspects  of  ATC  tasks,  such  as  the 
choice  of  symbols  for  use  in  ATC  labels  (David  and  Thomas,  1996).  Such  studies  must 
be  taken  with  some  reserve,  since  it  is  rarely  clear  how  such  items  are  used  in  practice, 
but  they  may  be  used  to  identify  what  subsequently  appear  to  be  obviously  unsuitable 
choices.  In  the  past,  we  have  used  standard  IBM  PCs  with  VGA  colour  screens  as  test¬ 
beds,  writing  the  necessary  display  software  in  QuickBASIC,  for  speed  and  convenience. 
Part-task  studies  of  this  type  should  take  from  four  to  six  weeks,  and  require  the 
cooperation  of  controllers,  or  non-controllers  for  about  one  half  day  per  individual.  Since 
these  part-tasks  usually  require  basic  perceptual  abilities  rather  than  integrated  ATC 
skills,  untrained  volunteers  can  usually  be  substituted  for  controllers  -  always  a  scarce 
resource. 

Demonstrators 

Demonstrators  are  small,  self-contained  programs  developed  to  demonstrate  new 
concepts,  or  to  afford  practice  in  unfamiliar  methods.  They  are  usually  written  in 
relatively  high-level  languages  and  presented  on  PC  compatibles.  Training  versions  can 
be  sent  to  participant  controllers  before  they  come  to  a  simulation  (as  was  done  with 
ODID  rV),  or  copies  can  be  distributed  to  provide  the  opportunity  for  ‘hands-on’ 
experience  with  EEC  Reports.  For  example,  a  3.5  inch  disk  containing  three 
demonstrators  is  distributed  with  EEC  Report  No,  307  (David,  1997a).  These  allow 
readers  to  compare  models  of  the  current  system  (DEMOLD),  a  radically  revised  system 
(DEMON)  and  a  more  tightly-coupled  predictive  system  (DEMFAST). 

More  elaborate  demonstrators  are  available  at  the  Experimental  Centre,  for  example,  to 
demonstrate  HIPS  displays  (Meckiff  and  Gibbs,  1994). 

Demonstrators  are  usually  constructed  by  one  or  two  persons  during  a  few  weeks.  They 
may  be  employed  in  formal  experiments  taking  some  months,  but  they  are,  more  often, 
used  to  demonstrate  features  of  planned  or  actual  systems. 

Conclusions 

This  brief  outline  shows  the  variety  of  simulation  methods  that  are  available  at  the 
Eurocontrol  Experimental  Centre.  Some  of  these  methods  involve  the  active 
participation  of  skiUed  controllers  (Real-Time  simulation,  TRACON/Pro),  others  are 
purely  computer  based  (Fart-Time  Simulation,  SIMMOD,  MicroSAINT),  complemented 
by  measirements  of  task  times  from  isolated  real  activities  (Part-Task  simulation,  Rapid 
Prototyping.)  Each  type  of  simulation  is  appropriate  to  particular  problems,  and  it  is 
only  by  an  intelligent  use  of  different  methods  that  a  fair  approximation  to  reality  can  be 
achieved. 

It  should  not  be  forgotten,however,  that  even  the  best  simulation  cannot  guarantee  that  a 
system  will  pass  the  ultimate  test,  that  of  reality. 
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Abstract 

To  meet  the  needs  of  growing  air  traffic,  new  air  traffic  control  (ATC)  systems  are  being 
constantly  developed.  Training  air  traffic  controllers  to  operate  these  systems  requires  a 
lot  of  time,  because  the  controllers  must  be  familiarised  with  both  new  human-machine 
interfaces  and  modified  operational  procedures.  For  that  reason,  the  National  Aerospace 
Laboratory  (NLR)  of  the  Netherlands  decided  to  conduct  a  study  into  the  feasibility  of 
computer  based  training  (CBT)  programs  running  on  PCs  as  a  means  of  familiarisation 
training  for  air  traffic  controllers.  By  presenting  the  familiarisation  training  prior  to  high 
fidelity,  operational  ATC  training,  it  was  expected  that  air  traffic  controllers  would  be 
better  prepared  for  the  latter  and  thus  require  less  operational  training  time.  During  the 
study  a  PC-based  tutorial,  as  well  as  a  PC-based  simulation  training,  were  developed.  As 
a  basis  of  the  training  modules  the  Converging  Runways  Display  Aid  was  chosen.  After 
performing  a  training  analysis  and  designing  the  structure  and  human-machine  interface, 
both  modules  were  implemented  using  Authorware.  Subsequently,  the  training  modules 
were  evaluated  by  operational  air  traffic  controllers.  From  the  study  it  can  be  concluded 
that  application  of  PC-based  familiarisation  training  for  air  traffic  controllers  is  feasible 
and  that  the  PC  is  an  appropriate  platform  for  this  kind  of  training. 

Introduction 

To  meet  the  needs  of  ever-growing  air  traffic  and  to  restrict  the  workload  of  the  air  traffic 
controller  to  acceptable  levels,  new  air  traffic  control  (ATC)  systems  are  being  constantly 
developed.  Before  these  systems  can  be  put  into  use,  air  traffic  controllers  must  be 
trained  to  operate  them.  Such  training  usually  comprises  familiarising  air  traffic 
controllers  with  new  human-machine  interfaces,  changed  task-allocation  between  human 
and  machine  (as  well  as  between  controllers  themselves),  and  modified  working 
procedures.  This  results  in  unacceptably  long  training  periods.  In  addition,  ATC  training 
programs  often  require  high  fidelity  ATC  simulation  equipment,  which  is  in  general 
costly,  scarce,  and  complex  to  organize. 

Hence,  NLR  decided  to  conduct  a  study  into  the  feasibility  of  a  so-called  computer  based 
training  (CBT)  running  on  personal  computers  as  a  means  of  familiarisation  training  for 
air  traffic  controllers.  This  PC -based  familiarisation  training  sought  to  introduce  air 
traffic  controllers  to  the  new  ATC  system.  By  presenting  the  PC-based  familiarisation 
training  prior  to  the  operational,  high  fidelity  ATC  simulator  training,  it  was  anticipated 
that  air  traffic  controllers  would  be  better  prepared  for  the  latter,  thus  reducing  the  time 
and  costs  associated  with  operational  training. 
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During  the  feasibility  study  a  PC -based  familiarisation  training  was  built  and  evaluated. 
This  training  consisted  of  two  modules:  a  PC-based  tutorial^  and  a  PC-based  simulation 
training^  As  a  basis  of  the  familiarisation  training  an  existing  ATC  system  was  chosen: 
the  Converging  Runways  Display  Aid  which  will  be  introduced  at  Amsterdam  Airport. 

The  Converging  Runways  Display  Aid 

To  increase  the  landing  capacity  of  Amsterdam  Airport  during  peak  traffic  hours,  ATC 
switches  from  a  single  runway  to  a  dual  simultaneous  runway  configuration.  Depending 
on  the  wind  conditions  (and  thus  landing  directions)  these  two  runways  may  converge.  A 
disadvantage  of  the  latter  is  that  during  poor  visibility  it  can  be  dangerous  to  land  aircraft 
on  both  runways  simultaneously.  Under  such  circumstances  the  controllers  in  the  control 
tower  cannot  directly  observe  the  in-trail  separation  between  arriving  aircraft.  This  in¬ 
trail  separation  is  of  vital  importance,  because  in  case  two  aircraft  break  off  their  landing 
one  after  the  other  and  make  a  climb,  the  danger  exists  that  they  will  collide  at  the 
intersection  of  both  runways.  To  mitigate  this  danger,  the  Converging  Runways  Display 
Aid  (CRDA)  is  used  to  set  up  a  safety  buffer  between  two  aircraft.  To  do  this,  the 
concept  of  ghosting  is  used.  Ghosting  refers  to  replicating  the  on-screen  image  of  an 
aircraft  at  a  different  spot  on  the  same  radar  display.  Such  a  copy  of  an  aircraft  is  called 
“ghost”  (see  figure  1).  The  CRDA  is  the  software  tool  that  generates  the  ghosts  and 
shows  them  on  the  radar  display. 

One  of  each  pair  of  converging  runways  is  designated  the  “Master  Runway”,  whereas  the 
other  is  designated  the  “Slave  Runway”.  The  Master  Runway  is  the  runway  from  which 
ghosts  are  copied.  The  Slave  Runway  is  the  runway  to  which  ghosts  are  copied.  Of  every 
aircraft  that  approaches  the  Master  Runway,  a  copy  is  shown  on  the  (extended  centre  line 
of  the)  Slave  Runway.  This  is  done  by  drawing  an  imaginary  circle  centred  on  the 
runway  intersection  from  the  actual  aircraft  to  the  Slave  Runway.  At  the  intersection  of 
the  circle  and  the  extended  centre  line  the  ghost  is  displayed.  As  the  actual  aircraft 
approaches  the  Master  Runway,  the  ghost  approaches  the  Slave  Runway  as  well;  that  is, 
the  ghost  and  its  original  behave  identically. 

Master  Arrival  and  Slave  Arrival  controller 

Both  the  Master  and  Slave  Runway  have  their  own  air  traffic  controllers.  The  air  traffic 
controller  of  the  Slave  Runway  is  called  “Slave  Arrival”,  whereas  the  air  traffic  controller 
of  the  Master  Runway  is  called  “Master  Arrival”.  On  the  Master  Arrivars  radar  display 
only  real  aircraft  are  displayed,  whereas  the  radar  display  of  the  Slave  Arrival  also  shows 
ghosts.  The  Master  Arrivars  task  is  to  direct  air  traffic  to  the  Master  Runway  in  such  a 
way  that  the  separation  between  the  aircraft  remains  constant.  To  prevent  collisions  at  the 
runway  intersection,  the  Slave  Arrival  has  to  direct  his  /  her  traffic  at  a  specific  minimal 
distance  behind  the  ghosts.  This  minimal  distance  is  the  above-mentioned  safety  buffer. 
Since  estimating  the  distance  between  aircraft  on  a  radar  display  is  perceptually  difficult, 
it  has  been  decided  to  put  the  ghosts  at  the  minimal  distance  behind  their  initial 
positions.  In  this  case  the  Slave  Arrival  only  has  to  direct  his  traffic  onto  or  immediately 

^  PC-based  tutorials  present  to  be  learned  knowledge  to  the  trainee,  ask  him  questions  about  this  knowledge,  evaluate 
his  answers,  and  present  him  feedback  about  the  correctness  of  these  answers. 

^  In  a  PC-based  simulation  training  trainees  acquire  skills  in  solving  problems  and  executing  working  procedures.  This  is 
done  by  simulating  a  part  of  the  real  world  and  letting  the  trainee  interact  with  it.  Each  time  the  trainee  interacts  with  the 
simulation,  the  latter  responds  by  changing  its  state,  so  that  the  results  of  the  trainee’s  actions  are  presented  to  him. 
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behind  the  ghosts  to  prevent  collisions  (see  figure  2),  The  process  of  directing  traffic  onto 
or  behind  ghosts  is  called  “tying”. 


Figure  1:  ghosting  Figure  2:  ghosting  with  safety  buffer 

Training  analysis 

To  develop  a  PC -based  familiarisation  training  based  on  the  CRDA,  a  training  analysis 
was  performed  first.  The  analysis  phase  of  the  CBT  development  consisted  of  four  steps: 

1 .  Select  the  air  traffic  controller  position  (Slave  Arrival  or  Master  Arrival)  to 
be  used  as  a  basis  for  the  training. 

2.  Perform  a  scenario  based  task  analysis  (i.e.  determine  which  tasks  the 
selected  air  traffic  controller  position  performs  in  which  operational 
scenarios). 

3.  Determine  the  skills  that  are  related  to  these  tasks  (i.e.  determine  what 
actually  has  to  be  trained). 

4.  Formulate  the  learning  objectives  (i.e.  describe  what  the  trainee  should  have 
learned  after  the  training). 

This  top-down  approach  ensured  that  no  necessary  skills  were  missed,  and  also  facilitates 
consultations  with  the  customer  (ATC  Amsterdam  Airport),  as  it  relates  closely  to  their 
current  training  practices. 

Selection  of  air  traffic  controller  position 

The  very  first  step  in  the  training  analysis  was  selection  of  a  CRDA  air  traffic  controller 
position  as  a  basis  for  the  CBT,  Because  the  project  was  intended  as  a  feasibility  study,  it 
was  decided  to  base  the  CBT  on  the  air  traffic  controller  position  that  would  benefit  the 
most  from  the  training. 

This  resulted  in  the  selection  of  the  Slave  Arrival  controller  position.  There  were  three 
reasons  for  this  decision: 

1.  The  tasks  of  the  Slave  Arrival  are  mainly  “new”  tasks  (i.e.  tasks  that  are  not 
part  of  the  regular  operations  of  the  controller). 

2.  The  correct  functioning  of  the  Slave  Arrival  is  crucial  for  the  success  of  the 
CRDA  operation, 

3.  The  Slave  Arrival’s  function  requires  the  highest  effort  from  the  controller. 
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Task  analysis 

The  next  step  in  the  training  analysis  was  to  determine  the  tasks  performed  by  the  Slave 
Arrival.  According  to  the  air  traffic  control  of  Amsterdam  Airport,  there  are  4  operational 
scenarios  related  to  the  Slave  Arrivars  function,  each  of  which  have  implications  for  PC- 
based  training.  These  four  scenarios  are: 

A.  Traffic  does  not  cause  any  problems:  normal  flow  of  traffiic. 

B.  Missed  approach  above  750 feet. 

A  missed  approach  is  a  runway  approach  that  is  broken  off.  In  this  scenario,  the 
aircraft  performs  the  missed  approach  above  an  altitude  of  750  feet. 

C.  Worst-case  situation  with  two  missed  approaches. 

A  worst-case  situation  is  a  situation  in  which  the  minimal  safety  buffer  would  be 
jeopardized  when  two  aircraft  (one  on  the  Master  Runway,  one  on  the  Slave 
Runway)  are  at  the  runway  intersection.  If,  during  such  a  worst-case  situation, 
both  aircraft  successively  initiate  a  missed  approach,  unpredictable  variations 
in  aircraft  behaviour  could  easily  lead  to  a  critical  loss  of  separation. 

D.  Excessive  approach  speed. 

If  the  approach  speed  of  an  aircraft  is  too  high,  the  chance  of  a  double  missed 
approach  (and  hence  the  danger  of  a  collision  on  the  runway  intersection)  is  no 
longer  minimal. 

For  each  of  the  operational  scenarios,  a  task  analysis  was  performed.  The  tasks  the  Slave 
Arrival  performs  in  these  scenarios  can  be  summarised  as  follows: 

•  Monitoring  whether  an  aircraft  is  under  his  /  her  control. 

•  Issuing  speed,  altitude  and  heading  instructions  to  the  aircraft  in  order  to 
direct  it  to  the  correct  position  for  tying. 

•  Monitoring  whether  the  tie-process  proceeds  correctly. 

•  When  an  aircraft  initiates  a  missed  approach,  executing  one  of  a  number  of 
so-called  “default  solutions”  to  prevent  a  collision  on  the  runway  intersection. 
The  default  solution  to  be  chosen  depends  on  the  runway  combination  used. 

•  Redirecting  aircraft  that  had  a  missed  approach  back  into  the  “main”  stream 
of  aircraft  to  approach  the  Slave  Runway  again. 

Training  skills 

The  third  step  in  the  analysis  was  to  use  the  identified  tasks  to  determine  the  skills  to  be 
trained.  A  training  skill  is  defined  as  the  product  “task  x  task  context  x  time  pattern”. 
The  context  of  the  tasks  at  Amsterdam  Airport  was  defined  by  the  runway  combinations 
used  (which  depend  on  wind  conditions),  and  the  type  of  aircraft  being  involved  in  the 
CRDA  operation.  Because  the  number  of  aircraft  per  hour  is  roughly  constant  when 
CRDA  is  in  operation  (as  the  CRDA  is  only  used  during  peak  traffic  hours),  the  time 
pattern  (the  time  available  for  each  task)  is  constant.  Therefore,  the  skiUs  are  a  result  of 
the  product  “task  x  task  context”.  For  the  four  given  scenarios,  this  resulted  in  60 
different  training  skills  (15  tasks,  4  different  task  contexts). 

Since  the  trainee  population  comprises  skilled  air  traffic  controllers,  not  all  training  skills 
identified  need  to  be  trained.  Certain  skills  (e.g.  ATC  -  aircraft  communication,  updating 
aircraft  labels)  would  likely  already  have  been  mastered  by  the  Slave  Arrival. 
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To  determine  which  of  the  training  skills  would  be  suitable  for  training  with  CBT,  it  was 
decided  to  add  each  of  these  to  the  CBT  one  by  one,  starting  with  the  simplest  scenario, 
scenario  A. 

Learning  objectives 

Finally,  the  last  step  in  the  training  analysis,  was  to  write  learning  objectives  based  on  the 
training  skills.  The  formulation  of  the  learning  objectives  was  based  on  recommendations 
by  Hannafm  and  Peck  (1988),  which  include  the  following: 

•  Each  learning  objective  should  contain  the  behaviour  to  be  shown  by  the 
trainee. 

•  Each  learning  objective  should  contain  the  circumstances  under  which  the 
behaviour  should  be  shown. 

•  Each  learning  objective  should  contain  the  criteria  the  student  behaviour 
should  satisfy. 

An  example  learning  objective  for  scenario  C  is  the  following: 

"’'The  trainee  is  able,  for  runway  combinations  06/0 IR  and  27/19R  and  air  traffic 
consisting  of  heavy,  medium  and  light  aircraft,  to  check  using  radar  display  and 
aircraft  labels,  whether  the  tie-process  is  proceeding  correctly,  and  whether  a 
worst-case  situation  is  caused'. 

However,  it  became  clear  that  learning  objectives  like  these  are  difficult  to  read,  and  not 
suitable  for  presentation  to  the  trainees.  Therefore,  the  objectives  were  reformulated.  For 
the  given  example,  this  resulted  in: 

""The  trainee  is  able,  for  runway  combinations  06/01R  and  27/19R,  to  check 
whether  traffic  is  causing  a  worst-case  scenario". 

Selecting  the  type  of  computer  based  training 

The  learning  objectives  could  be  classified  into  two  categories:  those  related  to  the  basic 
knowledge  of  CRD  A,  and  those  related  to  the  procedures  and  tasks  to  be  executed  by  the 
Slave  Arrival  during  the  identified  scenarios. 

For  the  learning  objectives  on  CRDA  knowledge,  it  was  decided  to  develop  a  PC-based 
tutorial  module.  Historically,  CRDA  knowledge  training  has  been  offered  classroom  style 
(with  paper  and  pencil).  Therefore,  the  PC-based  tutorial  uses  a  sequence  of  screens  with 
CRDA  information  (e.g.  when  is  CRDA  used,  under  which  circumstances),  interspersed 
with  “yes/no”-type  questions. 

It  has  been  known  for  well  over  ninety  years  that  transfer  of  learned  knowledge  and  skills 
from  training  to  the  real  world  is  only  achieved  if  the  training  situation  and  the  real  world 
contain  identical  elements  (Thorndike  &  Woodworth,  1901).  Therefore,  to  achieve  a  high 
positive  transfer  of  the  knowledge  and  skills  acquired  during  the  familiarisation  training 
to  the  real  world,  it  was  decided  to  use  simulation  training  to  train  the  procedural 
learning  objectives. 

By  means  of  a  simulation  training  the  radar  display  as  used  by  the  Slave  Arrival  could  be 
imitated.  In  order  not  to  have  the  simulation  training  result  in  a  high  fidelity  simulator  of 
the  Amsterdam  Airport  ATC  system,  only  the  elements  relevant  to  the  CRDA  procedures 
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of  the  Slave  Arrival  were  included  in  it.  The  input  of  the  instructions  to  the  aircraft 
would  have  to  be  as  identical  to  the  real  world  as  possible,  except  for  the  radio  telephony 
inputs.  The  PC-based  simulation  training  was  intended  to  be  offered  after  completion  of 
the  PC -based  tutorial. 

The  PC-based  tutorial 

For  the  tutorial,  the  CRDA-manual  was  chosen  as  a  starting  point.  From  this  manual,  the 
information  that  should  be  known  to  the  Slave  Arrival  before  he  /  she  is  able  operate  the 
CRDA  was  extracted.  This  information  was  organized  in  a  number  of  topics,  like 
“CRDA  terms  and  definitions”,  and  “Specific  Schiphol  Airport  CRDA  procedures”. 

The  information  on  each  of  these  topics  was  organized  in  a  number  of  screens  containing 
information.  Where  appropriate  simple,  non-interactive  animations  were  added  to 
explain  some  terms.  For  example,  the  difference  between  ghosting  with  and  without 
safety  buffer  was  demonstrated  using  two  animations.  In  these  animations,  all 
unnecessary  details  were  left  out,  so  that  the  trainee  could  concentrate  on  the  example 
given. 

The  topics  were  interspersed  with  questions  in  which  the  trainee  had  to  indicate  whether 
the  statement  made  in  the  question  was  correct  (“yes/no”  questions).  A  trainee  could  only 
proceed  to  the  next  topic  if  all  questions  in  a  block  were  answered.  For  each  question  that 
was  answered  incorrectly,  the  trainee  was  presented  feedback  indicating  why  the  answer 
was  incorrect.  For  some  questions,  there  was  additional  clarifying  feedback  after  correct 
answers  as  well. 

Additional  requirements  to  the  simulation  training 

Following  the  decision  to  develop  a  PC-based  tutorial,  as  well  as  a  PC-based  simulation 
training,  additional  requirements  on  the  simulation  training  were  defined.  These  were  of 
the  following  categories: 

•  Requirements  with  respect  to  the  structure  of  the  simulation  training.  It  was 
decided  that  the  simulation  training  should  consist  of  a  familiarisation  lesson 
in  which  the  trainee  can  become  familiar  with  the  radar  maps  of  both  CRDA 
runway  combinations  and  with  the  procedures  for  issuing  aircraft  instructions. 
Further,  a  lesson  was  required  in  which  trainees  could  practise  tying  air  traffic 
on  both  runway  combinations  in  each  of  the  operational  scenarios. 

•  Requirements  with  respect  to  information  shown  to  the  trainee.  For  instance, 
update  rate  of  aircraft  positions,  shape  of  aircraft  position  symbols,  content  of 
aircraft  labels  (altitude,  speed,  etc.),  layout  of  radar  maps,  colours. 

•  Requirements  with  respect  to  interactions  between  trainee  and  training.  For 
example,  how  to  select  an  aircraft  in  order  to  issue  instructions  to  it,  how  to 
input  aircraft  instructions,  how  to  update  aircraft  label  data. 

•  Requirements  with  respect  to  the  behaviour  of  simulated  aircraft.  Such  as 
rate  of  turn,  rate  of  climb,  rate  of  descent,  runway  approach  routes,  initial 
aircraft  separation. 
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Design  of  the  simulation  training 

Based  on  the  learning  objectives  and  additional  requirements,  a  design  of  the  simulation 
training  was  made.  Figure  3  shows  the  design  of  the  structure  of  the  simulation  training. 


Figure  3:  design  of  the  structure  of  the  simulation  training 

The  simulation  training  took  off  with  an  introduction  that  briefly  described  the  content  of 
the  training.  After  the  introduction  the  trainee  could  follow  the  familiarisation  or  the 
tying  lesson.  On  request  of  the  trainee  the  familiarisation  lesson  presented  explanations 
about  elements  of  either  of  the  radar  maps.  In  this  lesson  the  trainee  could  also  practise 
issuing  instructions  to  aircraft,  hi  the  tying  lesson  the  trainee  could  practise  tying  aircraft 
on  both  CRDA  runway  combinations  (06/01R  and  27/19R)  and  in  each  of  the  four 
scenarios. 

In  addition  to  the  structure  of  the  simulation  training,  a  design  of  the  interface  between 
trainee  and  training  was  made.  This  interface  design  was  based  on  the  DenK  architecture 
(Ahn  et  al.,  1994).  In  accordance  with  the  DenK  architecture  the  interface  between 
trainee  and  training  can  be  illustrated  as  shown  in  figure  4. 


Figure  4:  interface  between  trainee  and  simulation  training 

The  simulation  training  embodied  two  elements:  a  domain  and  a  tutor.  The  purpose  of 
the  domain  was  to  simulate  the  real  world  in  which  the  Slave  Arrival  performs  his  /  her 
tasks.  Depending  on  the  training’s  state,  the  domain  could  be  of  different  types.  For 
instance,  if  the  simulation  training  was  in  the  state  Radar  map  06/01R  (see  figure  3), 
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then  the  domain  contained  a  picture  of  the  radar  map,  as  shown  on  the  Slave  Amval  s 
radar  display  during  CRD  A  operations  for  runway  combination  06/0 IR.  When  the 
training  was  in  the  state  Familiarisation  aircraft  manipulation  the  domain  held  the 
aforementioned  radar  map,  as  well  as  simulated  air  traffic  and  an  input  field  to  issue 
aircraft  instructions.  The  domain  interacted  with  the  tutor  by  for  example  supplying  it 
with  trainee  performance  data.  The  trainee  interacted  with  the  domain  by  requesting 
explanations  about  radar  map  elements  and  by  issuing  instructions  in  order  to  change 
aircraft  behaviour. 

On  the  trainee’s  request  the  tutor  showed  help-info  and  interacted  with  the  domain  to 
start  and  end  lessons  and  domain  types.  Also,  the  tutor  read  in  and  verified  aircraft 
instructions  issued  by  the  trainee.  Based  on  performance  data  received  from  the  domain 
the  tutor  also  presented  the  trainee  feedback  about  the  correctness  of  his  /  her  behaviour. 
Cohen  (1985)  and  Sales  (1988)  say  that  such  feedback  is  both  motivating  and 
informative.  This  means  that  the  feedback  attempts  to  alter  the  trainee’s  behaviour  by 
rewarding  correct  behaviour  by  praising  him  /  her  and  by  discouraging  incorrect 
behaviour  by  telling  him  /  her  about  mistakes  and  their  consequences.  It  also  means  that 
the  discouraging  feedback  offers  the  trainee  information  about  where  the  mistakes  were 
made  and  how  they  can  be  corrected.  Furthermore,  the  training  presented  feedback  about 
the  degree  to  which  the  trainee  was  reaching  the  learning  objectives.  The  training  showed 
for  instance  the  number  of  correctly  tied  aircraft  and  the  utilised  landing  capacity. 
According  to  Latham  and  Locke  (1979)  this  feedback  type  improves  the  trainee’s 
commitment  to  the  learning  objectives. 

Another  educational  concept  that  has  been  included  in  the  PC-based  simulation  training 
is  the  concept  of  overtraining.  Overtraining  means  presenting  trainees  additional 
learning  opportunities  after  they  have  reached  the  learning  objectives.  When  applied, 
overtraining  improves  the  retention  of  acquired  knowledge  and  skills  (see  Schendel  & 
Hagman,  1982).  The  concept  of  overtraining  was  included  in  the  simulation  training  by 
restarting  the  tying  lesson  whenever  the  trainee  succeeded  for  the  first  time  in  tying  all 
aircraft  offered  to  him  /  her  during  the  lesson. 

Implementation  of  the  CBT 

For  the  implementation  of  the  CBT  modules,  Authorware  Professional  was  used.  For  the 
tutorial,  no  major  problems  were  met.  Nevertheless,  for  the  simulation  training  it  became 
clear  that  a  real-time  interactive  simulation  required  a  number  of  adjustments  to  be  made, 
especially  in  order  to  keep  the  simulation  running  during  trainee  interactions.  Further,  it 
proved  impossible  to  make  a  scaleable  simulator;  that  is,  a  simulator  in  which  the 
number  of  Master  and  Slave  aircraft  can  be  changed  easily.  For  training  purposes  though, 
there  would  be  a  fixed  maximum  number  of  aircraft  on  the  radar  display  at  a  given  time, 
so  this  problem  was  more  or  less  circumvented. 

Figure  5  shows  a  sample  screen  image  containing  the  main  radar  screen  of  the  simulation 
training. 

Evaluation  of  the  PC-based  tutorial  and  PC-based  simulation  training 

After  they  had  been  implemented,  the  PC-based  tutorial  and  PC-based  simulation 

training  were  evaluated  by  air  traffic  controllers  from  Amsterdam  Airport.  These 
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controllers  were  asked  to  attend  the  tutorial  and  simulation  training  and  to  offer  feedback 
on  each  of  them.  After  correction  of  all  shortcomings  that  were  found  in  the  modules,  a 
questionnaire  was  developed  in  which  controllers  were  requested  to  rate  the  tutorial  and 
simulation  training  on  a  number  of  topics,  including  aircraft  behaviour,  feedback,  and 
help-information.  The  intention  was  to  deploy  both  modules  during  the  official  CRDA 
training  of  the  Dutch  air  traffic  control  services  and  to  hand  out  the  questionnaire  among 
the  trainees  attending  the  latter.  However,  due  to  circumstances  beyond  the  control  of 
NLR,  dates  have  not  yet  been  finalised  for  this  empirical  evaluation. 


Figure  5:  sample  screen  image  from  the  simulation  training 


Conclusions 

Even  though  the  PC-based  training  modules  have  not  been  empirically  evaluated  yet,  the 
software  developed  so  far,  and  the  remarks  Ifom  Amsterdam  Airport  air  traffic 
controllers  indicate  that  the  application  of  PC -based  training  for  familiarisation  training 
of  air  traffic  controllers  is  feasible,  and  that  the  PC  is  an  appropriate  platform  for  this 
type  of  training. 

During  the  study  it  was  experienced  too  that  a  formal  approach  to  training  analysis 
assists  in  determining  the  skills  and  learning  objectives  to  be  satisfied  with  the  training, 
as  well  as  supports  consultations  between  developer  and  customer. 

Furthermore,  the  feasibility  study  revealed  that  application  of  Authorware  Professional  to 
develop  real-time  simulations  is  not  optimal.  Instead,  it  might  be  better  to  create  external 
simulation  modules,  which  could  be  integrated  into  the  Authorware  training- software. 
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Abstract 

Many  large-scale  dynamic  control  systems  are  moving  from  traditional  equipment- 
determined  control  systems  to  computer-mediated  wholly  or  semi-automated  systems. 
There  has  been  a  tendency  for  computer-based  systems  to  mimic  traditional  displays. 
This  approach  fails  to  make  use  of  the  enhanced  information-handling  capacity  of  the 
computer,  and  often  produces  displays  that  are  in  practice  inferior  to  those  they  replace  - 
occasionally  with  tragic  results, 

A  better  approach  is  to  begin  with  an  qualitative  functional  analysis  of  the  control  task, 
with  as  little  reference  to  the  existing  system  as  possible,  then  to  consider  the  component 
tasks,  selecting  those  for  which  the  operators  are  best  suited  to  compose  an  interesting 
but  not  overloading  workload,  and  delegating  the  remaining  tasks  to  the  computer-based 
system.  Where  the  workload  varies  with  time,  particularly  if  the  variation  is 
unpredictable  or  uncontrollable,  it  may  be  necessary  to  provide  ‘fall-back’  automation, 
which  will  maintain  safety  without  regard  to  efficiency. 

Only  after  the  operators’  overall  tasks  have  been  defined  should  consideration  be  given  to 
what  and  how  to  display  information  to  the  operators,  and  to  the  communication  links  to 
be  used.  In  many  instances  graphical  images  will  be  preferable  to  tabular  and  symbohc  to 
numeric  forms.  Displays  need  not  be  confined  to  current  data  -  historical  trends  and 
projections  of  future  situations  can  be  displayed.  The  latter  can  be  particularly  valuable  if 
linked  with  projected  sequences  of  control  actions,  to  verify  their  effectiveness  before 
applying  them.  Where  control  is  largely  ‘by  exception’  displays  must  be  designed  to 
enable  the  operator  to  obtain  the  necessary  information  for  effective  control  in  adequate 
time.  Training  may  include  drills  for  the  allocation  of  tasks  between  operators,  as  with 
CRM  (Cockpit  Resource  Management)  in  civil  aviation. 

Introduction 

Dynamic  control  systems  control  processes  that  are  operating  in  ‘real-time’.  The  events 
or  processes  controlled  are  happening  now,  and  usually  continuously.  Examples  are 
chemical  plants,  assembly  plants,  automated  factories,  water  and  sewage  distribution 
systems,  electricity  generating  plants  (nuclear,  coal,  hydroelectric  or  others),  electricity 
distribution  systems,  road,  rail  and  air  traffic  control  systems,  ambulance  services,  and 
even  military  command  systems. 

These  systems  usually  have  control  rooms,  where  operators  ,  usually  in  teams,  receive 
information,  (as  verbal  messages,  by  radio,  telephone  or  data-link,  TV  images,  readings 
on  dials  or  chart-recorders,  warning  lights,  gauges,  or  simply  by  looking  out  of  the 
window)  form  a  picture  of  the  activity  (which  may  be  literally  a  picture,  some  form  of 
semi-mechanical  analog  display,  a  computer  record,  or  other  display),  decide  on  the 
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necessary  control  actions  required,  and  communicate  these  to  the  components  of  the 
system,  by  whatever  means  available  (Button  pushing,  verbal  messages,  maneuvering 
controls  etc.).  Finally, (and  often  forgotten),  they  must  check  that  their  intended  actions 
have  actually  happened,  and  that  they  have  had  the  desired  effect. 

Many  of  these  systems  are  made  up  of  more  or  less  autonomous  units.  The  degree  of 
centralization  or  delegation  of  authority  is  usually  the  result  of  historical  or  technical 
accident  rather  than  design.  Although  the  general  problems  are  similar,  accidents  of 
technical  development  have  led  to  different  styles  of  display  in  different  fields  of 
development.  Ergonomists  have  spent  much  time  and  effort  in  the  design  of  ‘knobs  and 
dials’.  The  studies  by  Fitts  and  Jones(i961)  are  classic  examples. 

There  is,  however,  a  contemporary  trend  that  must  inevitably  affect  the  design  of  control 
rooms.  More  and  more  of  these  systems  are  computer-based.  Rather  than  displaying  raw 
radar  data,  the  signals  from  several  radars  are  smoothed  and  combined,  weighted 
according  to  the  reliability  of  the  individual  radar  and  used  to  construct  a  digital  track  for 
an  aircraft,  which  is  then  displayed  on  a  computer  VDU.  (My  life’s  experience  is  in  Air 
Traffic  Control,  so  I  shall  draw  examples  mainly  from  that  field,  but  the  message  I  wish 
to  send  applies  to  all  the  fields  I  have  mentioned,  and  to  many  others.  Hansen  (1995) 
provides  a  similar  example  for  a  coal-fired  power  station.  )  The  circular  display  with 
rotating  sweep  so  familiar  from  films  is  now  unnecessary,  and  a  standard  CRT  can  be 
used.  Ironically,  some  effort  has  made  to  produce  a  synthetic  digital  representation  of  the 
trail  of  decaying  ‘blips’  behind  the  aircraft. 

The  Horseless  Carriage  Method 

This  is  an  example  of  the  ‘horseless  carriage’  effect.  In  order  to  ease  perceptual  problems 
in  the  transition  from  traditional  electromechanical  to  digital  systems,  it  is  tempting  to 
reproduce  the  traditional  image  on  the  CRT  screen.  This  may  have  disastrous 
consequences.  The  image  available  on  a  flat  CRT  screen  may  be  far  less  salient  than  that 
provided  by  a  bank  of  electromechanical  dials,  so  that  minor  but  critical  differences  can 
be  overlooked.  A  more  subtle  problem  occurs  in  Air  Traffic  Control,  where  the 
traditional  planner  is  provided  with  a  strip,  in  a  strip-holder,  for  each  aircraft.  He  moves 
these  strips  around,  marks  the  surface  of  the  strip,  moves  it  in  its  holder  to  signal  to  his 
partner,  and  finally  discards  it  and  rearranges  his  board  after  the  aircraft  has  left  his  area 
of  responsibility.  Electronic  strips  lack  these  facilities,  and,  in  some  systems,  may  even 
remove  themselves  automatically  when  the  aircraft  leaves,  so  that  when  the  controller 
next  looks  at  his  strip  bay  the  aircraft  are  not  where  he  left  them.  This  is,  to  put  it  mildly, 
intolerable  to  a  busy  controller. 

Examples  abound  of  similar  short-sighted  adaptations  of  pre-computer  displays.  They 
have  some  superficial  attractions.  They  do  not,  in  principle,  require  significant  input 
from  the  users,  and  they  are  usually  sufficiently  close  to  the  previous  system  that  no 
training  or  transition  period  is  apparently  necessary.  Often  neither  of  these  is  true,  but,  by 
the  time  this  becomes  apparent,  the  system  is  in  place. 

Beyond  the  Interface 

I  propose  a  different  approach  to  the  development  of  these  systems.  None  of  the 
individual  elements  of  this  approach  is  unprecedented,  and  most  of  what  I  am  saying  can 


be  derived  from  various  studies,  ranging  from  neurophysiology  to  anthropology  to 
systems  analysis,  carried  out  in  the  last  twenty-five  to  thirty  years. 

Task  Definition 

The  starting  point  of  interface  design  should  be  a  careful,  in-depth  study  of  what  are  the 
real  purposes  of  the  control  system  being  investigated.  Most  organizations  have  a 
standard  definition,  which  will  be  produced  in  answer  to  the  initial  question.  In  Air 
Traffic  Control,  this  is  usually  ‘The  Safe,  Orderly  and  Economic  control  of  Air  Traffic”, 
for  example.  If,  however,  you  analyze  the  components  of  this  statement,  you  find  that  the 
first  is  a  constraint,  the  second  is  practical  requirement  of  pre-computerized  systems 
arising  from  the  limits  of  human  memory  and  i^ormation  processing  capacity,  and  the 
third  is  often  a  hope  rather  than  a  true  goal.  Although  it  is  always  possible  to  argue  over 
abstract  definitions,  ATC’s  purpose  can  be  defined  as  to  allow  air  traffic  to  fly  from 
origin  to  destination  as  quickly  and/or  cheaply  as  the  individual  flights  require,  with  the 
minimum  disturbance  necessary  to  maintain  safety. 

Task  Analysis 

The  second  stage  of  the  process  is  to  analyze  the  ‘real’  task.  This  may  well  not  be  closely 
related  to  the  formal  task  as  traditionally  defined.  It  is  particularly  important  to 
distinguish  between  a  task  ‘analysis’  and  a  task  ‘description’.  The  construction  of  a  task 
description  involves  observation  of  the  existing  system,  often  in  extraordinary  detail 
(Phillips  et  al,  1984),  has  the  unfortunate  consequence  of  producing  a  mental  ‘set’  in 
favor  of  the  existing  method  of  control.  For  simplicity  and  convenience,  I  shall  take  as 
example  ‘en-route’  ATC,  which  concerns  aircraft  in  the  major  part  of  their  flight,  after 
they  have  left  the  region  around  their  airport  of  origin,  and  before  they  arrive  at  their 
destination.  The  TMAs  (Terminal  Maneuvering  Areas)  around  airports  or  groups  of 
airports  have  their  own  problems,  which  will  not  be  discussed  here. 

A  modem  Upper  Airspace  Control  centre,  such  as  Maastricht  UAC,  where  Eurocontrol 
controls  the  upper  airspace  of  several  busy  states,  and  part  of  another,  is  organized  into 
sectors,  each  defined  by  a  geographical  area  and  upper  and  lower  flight  levels.  Within 
the  sectors,  routes  are  defined  by  sequences  of  beacons.  Ground-air  communication  is 
by  voice  links.  The  data  for  each  flight,  stating  its  intentions,  is  abstracted  from  standard 
flight  plans  and  combined  with  information  from  a  network  of  radars  to  provide  current 
aircraft  positions.  Although  Flight  Process  Strips  are  still  generated,  in  some  sectors 
controllers  prefer  to  refer  to  electronic  data  displays.  Usually  sectors  are  manned  by  two 
controllers.  The  Planning  Controller  (PC)  examines  the  future  traffic,  and  plans  how  it 
can  be  organized  to  minimize  the  disturbance  to  the  aircraft  while  maintaining  safe 
separations  and  satisfying  entry  and  exit  conditions.  The  Executive  Controller  (EC), 
referring  mainly  to  the  radar  picture,  makes  short-term  decisions,  and  communicates  with 
the  aircraft  in  the  sector.  Controllers  communicate  with  aircraft  by  Radio-telephony 
(RTF),  with  adjacent  sectors  and  centers  by  telephone  or  by  computer-based  digital 
messages,  and  with  each  other  by  voice,  supplemented  by  gestures  and  computer  or  strip 
notations.  In  all  modem  ATC  systems,  the  controllers  are  obliged  to  update  the  computer 
system  when  they  take  action,  and  in  most  modem  aircraft,  the  pilots  insert  the  changes 
in  their  flight  path  into  their  Flight  Management  System  (FMS). 
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Figure  1  (based  on  Dee  1996)  sketches  the  tasks  currently  carried  out  by  en-route  Air 
Traffic  Controllers. 

1)  learn’ sectors 

2)  manipulate  and  mark  strips 

3)  plan  future  streams  of  aircraft  entering  their  sectors, 

4)  check  that  they  will  not  conflict  with  each  other  within  the  sector, 

5)  determine  how  to  resolve  conflicts 

6)  match  radar  images  to  strips 

7)  acknowledge  aircraft  coming  on  to  the  frequency  when  they  enter, 

8)  intervene  to  resolve  conflicts 

9)  coordinate  their  actions  with  the  next  sector 

10)  monitor  aircraft  behavior  for  deviations  from  track 

1 1)  hand  aircraft  over  to  the  next  sector 

1 2)  attempt  to  comply  v^th  any  special  requests, 

(for  example  for  direct  routings  or  changed  routings), 

13)  handle  unexpected  emergencies. 

Figure  1  -  Task  List 

We  do  not  have  enough  space  and  time  to  carry  out  a  systematic  analysis  of  these  tasks 
(See  David  1997),  although  it  should  be  noted  that  the  first  mentioned  task  -  ‘learning 
sectors’  is  particularly  interesting  from  an  ergonomic  point  of  view,.  It  takes  a  fully- 
trained  controller  about  six  months  to  ‘learn’  a  sector.  Too  little  attention  has  been  paid 
to  this  process,  which  is  generally  one  of  apprenticeship.  The  controller  accompanies  an 
experienced  controller  on  the  sector,  apparently  absorbing  a  vast  quantity  of  information 
about  the  sector,  the  usual  traffic  and  its  peculiarities,  so  that  this  information  is  in  his 
permanent  memory  for  recall.  The  necessity  for  this  feat  of  memorization  is  a  criticism  of 
the  current  systems  of  training  and  information  presentation.  This  information  is  a 
considerable  burden  to  the  controller,  fading  rapidly  after  a  few  days’  interruption  of  the 
task.  Many  ‘coping  strategies’  are  required  to  maintain  it,  even  with  the  present  level  of 
traffic.  (Stein  and  Bailey,  1994) 

Many  of  the  tasks  listed  here  are  essentially  routine,  linked  to  the  traditional  methods  of 
communication  by  RTF,  occupying  a  considerable  proportion  of  the  controllers’  time. 
Others  reflect  the  need  to  match  the  strip  or  tabular  data  on  which  planning  is  based,  and 
the  decisions  arrived  at  during  planning,  to  the  image  of  the  current  situation  presented 
by  the  Electronic  Data  Display  (EDD)  on  which  the  annotated  radar  data  is  displayed. 
This  operation  is  inherently  wasteful,  since  the  computer  has  already  done  it.  Some 
modem  stripless  systems  (Graham  et  al,  1994),  eliminate  this  part  of  the  workload  by 
displaying  the  ‘strip’  data  as  part  of  an  aircraft  label.  This  approach  has  other  drawbacks 
-  it  covers  the  screen  with  tabular  data,  or  requires  separate  windows  to  be  opened  for 
each  aircraft. 

Communication 

The  communication  of  information  by  RTF  is  inherently  error-prone,  and  the  attempt  to 
achieve  mechanical  levels  of  reliability  by  drilling  human  operators  is  futile  and 
inhumane.  Cushing  (1994)  draws  on  reports  of  the  (US)  National  Traffic  Safety  Board, 
and  the  Aviation  Safety  Reporting  System,  to  show  that  this  form  of  communication  is 
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no  longer  viable.  He  sketches  a  potential  ‘data-link’  alternative. 

Human  Tasks 

To  summarize  an  extensive  argument,  the  tasks  particularly  suited  to,  and  enjoyed  by 
human  operators  are  conflict  resolution  (but  NOT  conflict  detection),  dealing  with 
special  requests  from  aircraft,  and  rectifying  errors  identified  by  computer  monitoring  of 
traffic,  such  as  may  arise  from  incorrectly  set  Flight  Management  Systems.  Controllers 
are  also  good  at  imposing  a  strategic  direction  on  the  system,  by  developing  and  applying 
conventions  to  reduce  workload  and  increase  overall  efficency. 

The  human-computer  interface  should  therefore  be  designed  to  facilitate  these  tasks, 
rendering  the  data  necessary  for  these  tasks  easily  accessible,  while  not  prohibiting  the 
controller  from  looking  further  into  the  data  when  opportunity  offers. 

As  Woods  (1995)  remarks,  the  now  traditional  windowing  structure  is  not  well  adapted 
for  dynamic  control,  since  it  requires  the  user  to  search  out  the  correct  window  to  find 
the  data  or  formulate  the  instruction  required.  It  is  far  better,  if  at  all  possible,  to  show  all 
necessary  information  on  one  screen.  This  may  at  first  seem  a  difficult  task,  but  it  is  not 
impossible,  if  the  nature  of  the  data  and  the  way  they  are  used  are  taken  into  account. 

The  data 

For  design  purposes,  we  can  simplify  the  actual  ATC  situation  by  considering  the 
trajectoiy  of  each  aircraft  as  a  series  of  linked  straight-line  segments.  Each  begins  with  a 
position  in  space,  expressed  as  X,  Y  and  H  coordinates,  and  a  (constant)  rate  of  change 
of  each  of  these.  The  position  is  true  at  a  time  T  and  the  segment  continues  until  one  or 
more  of  the  rates  of  change  alters  after  an  interval  dT.  Segments  before  the  current  time 
are  fixed,  and  of  no  immediate  interest  for  control  purposes.  Future  segments  describe 
the  planned  future  path  of  the  aircraft.  These  can  be  compared  with  the  future  segments 
of  other  aircraft  to  identify  losses  of  separation  at  future  times.  The  use  of  straight-Hne 
segments  permits  a  fairly  simple  algorithm  to  be  employed.  Changes  in  future  flight  path 
can  be  specified  simply,  and  converted  into  an  alternative  sequence  of  segments,  which 
can  be  tested  to  identify  new  or  remaining  conflicts. 

Traditional  displays 

There  are  two  traditional  means  of  displaying  this  data.  The  radar  image  displays  the 
current  X  and  Y  dimensions  as  positions  on  the  display  surface,  dX  and  dY  by  a  ‘speed 
vector’  and  codes  height  (H)  as  a  three-digit  numbers  displayed  close  to  the  aircraft 
marker,  in  a  ‘label’  containing  the  call-sign,  and  more  or  less  other  data  expressed  as 
letters  and  digits.  The  strip  display  (usually)  reduces  the  X  and  Y  dimensions  to  a  series 
of  named  points  (usually  beacons  where  routes  intersect)  for  which  estimated  times  (T) 
and  heights  (H)  are  specified  as  numeric  values.  The  strips  themselves  may  be  arranged 
in  various  ways,  for  example,  under  a  label  for  the  next  beacon,  arranged  by  height  or 
estimated  time,  where  the  labels  are  arranged  in  a  roughly  geographical  layout.  It  is  very 
difficult  to  cope  with  aircraft  which  do  not  follow  predefined  routes  using  a  strip-like 
display. 

Apart  from  the  position  and  speed  vector,  virtually  all  the  data  are  presented  in  letters 
and  digits.  To  complicate  matters  further,  beacons  are  usually  shown  by  abbreviations,  if 
they  are  labeled  at  all,  and  spoken  as  their  full  names.  Commercial  flight  call-signs 
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contain  a  company  identifier  such  as  PA  for  the  (defunct)  Pan-American  Airways,  which 
was  spoken  as  ‘Clipper’. 

From  the  cognitive  point  of  view,  this  is  a  mess.  If  the  theories  of  cerebral  asymmetry  of 
cognitive  function  (Springer  and  Deutsch,  1993)  have  any  validity,  it  would  be  hard  to 
design  a  worse  display  system.  Graphical  and  textual  indicators  are  scrambled  together 
unsystematically.  In  some  systems,  rates  of  climb,  known  to  the  computer  system  to  a 
high  degree  of  accuracy,  are  deducible  only  by  watching  the  rate  at  which  the  last  digit, 
representing  hundreds  of  feet,  changes.  If,  as  is  usually  the  case,  the  display  is  updated 
only  at  five  or  ten  seconds  intervals,  it  may  take  up  to  half  a  minute  to  obtain  even  an 
approximate  idea  of  the  climb  rate.  Controllers  adapt  to  this  by  learning  the  trajectories 
to  be  expected  of  common  types  of  aircraft  in  the  hands  of  particular  company  pilots,  and 
by  allowing  enormous  safety  margins  (Lafon-Millet  1978). 

Conflicts,  the  main  justification  for  en-route  air  traffic  control  are  usually  not  shown  to 
the  controller,  (Some  systems  have  last-moment  short-term  conflict  alerts,  but  many  of 
these  rely  only  on  radar  data,  so  that  they  cannot  cope  with  aircraft  which  have  been 
instructed  to  cease  climbing  below  the  level  of  the  opposing  traffic,  for  example,  and 
generate  many  false  alarms.)  In  the  ODID  (Prosser  et  A1  ,1991,  Graham  et  al  1995) 
research  simulations,  a  Conflict  Risk  Display  was  provided  which  consisted  of :  - 

•  A  rectangular  display  showing  time-to-go  as  the  horizontal  and  minimum  separation 
as  the  vertical  axis.  Numbered  points  identified  unsolved  conflicts.  A  line  represented 
the  duration  of  separation  loss. 

•  A  table  showing  the  call-signs  of  the  pairs  of  aircraft  involved  in  each  conflict. 

To  resolve  a  conflict,  the  controller  made  a  mental  note  of  the  number,  referred  to  the 
table,  made  a  mental  note  of  the  two  call- signs,  then  searched  the  radar  display  to  find 
the  two  aircraft  involved-  He  then  estimated  their  relative  speeds,  noted  their  heights, 
checked  or  recalled  their  future  trajectories,  estimated  their  relative  positions  at  the  time 
of  conflict.  (It  is  often  easier  to  solve  a  conflict  by  maneuvering  the  second  aircraft  to 
pass  the  crossing  point.)  Experienced  controllers  would  scan  the  table  of  call-signs  to  see 
if  one  aircraft  was  involved  in  several  conflicts,  and  search  for  a  maneuver  which  would 
solve  them  all.  This  display  was  greatly  liked  by  controllers.  It  may  be  that  this  was 
because  it  was  usually  blank,  assuring  them  that  that  they  had  not  missed  a  conflict  in 
their  normal  scanning  of  the  display,  a  constant  fear  of  all  controllers. 

Revised  display 

We  will  assume  that  we  use  a  map-like  display  as  our  primary  display,  showing  aircraft 
at  positions  by  symbols.  Figure  2  shows  a  representative  contemporary  and  a  revised 
aircraft  symbol. 
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Figure  2  -  Aircraft  image 


Figure  3  lists  the  aspects  displayed,  with  brief  notes  justifying  the  choices  of  display 
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Figure  3  -  Coding 


Height  is  indicated  by  colour  coding,  Eastbound  cruising  levels  are  shades  of  yellow, 
westbound  are  shades  of  blue.  Aircraft  in  level  flight  are  normally  at  a  fixed  level,  and 
are  coloured  accordingly.  Aircraft  climbing  or  descending  have  the  front  end  in  the  level 
they  are  approaching,  and  their  rear  in  that  they  are  leaving.  The  speed  indication  is 
highly  approximate,  but  is  rarely  significant  (see  below). 

Figure  4  shows  a  representative  screen,  as  displayed  on  a  rather  primitive  VGA  display. 
There  are  40  aircraft  in  random  flight  on  this  display.  Seventeen  of  these  aircraft  are 
involved  in  ten  potential  future  conflicts. 
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Figure  4  -  Revised  Display 

Figure  5  shows  the  potential  conflicts  present  in  this  image.  Conflicts  are  displayed  by 
linking  the  aircraft  involved.  This  leads  the  eye  to  the  aircraft  most  involved  in  conflicts. 
Lines  linking  conflicts  are  made  more  salient  as  the  time  to  go  before  loss  of  separation 
becomes  shorter.  Figure  5  also  shows  a  logical  extension  of  the  simplification  process, 
where  aircraft  that  are  not  involved  in  conflicts  are  not  shown  to  the  controller.  Many, 
possibly  most  aircraft  would  pass  through  the  airspace  without  contacting  the  controller 
at  all. 


(The  reader  is  invited  to  try  to  identify  the  aircraft  and  their  conflicts  before  looking  at 
Figure  5.) 
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Figure  5  -  Potential  Conflicts 

To  resolve  conflicts,  instructions  must  be  given  to  one  or  more  aircraft.  At  present,  these 
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are  given  by  the  Executive  Controller,  by  RTF,  The  EC,  operating  in  the  present,  gives 
instructions  to  be  executed  directly  by  the  pilot.  He  must  monitor  the  pilot’s  reply,  to  be 
sure  that  his  order  has  been  correctly  received.  He  must  check  a  minute  or  so  later  that 
the  aircraft  is  actually  doing  what  he  asked.  He  must  note,  or  remember,  when  he  has 
altered  the  direction  of  an  aircraft,  in  order  to  return  it  to  track  or  height  later.  He  must 
estimate  a  sufficient  diversion  from  track  to  respect  the  required  separation.  For  safety  s 
sake,  he  overestimates.  Where  the  separation  minimum  is  five  miles,  he  usually  adds 
“two  more  for  the  wife  and  kids”. 

Conflicts  are  often  detected  by  the  Procedural  controller,  when  comparing  strips.  He  may 
annotate  a  strip  to  tell  the  EC  to  move  the  aircraft  to  a  higher  or  lower  level  before  a 
certain  beacon.  Since  he  cannot  be  certain  exactly  when  the  EC  will  make  the  move,  both 
levels  may  be  blocked  for  a  considerable  time. 

Our  proposed  display  (Figure  5)  allows  the  controller  to  choose  which  aircraft  he  will 
maneuver,  but  the  system  can  be  set  up  to  choose  the  next  aircraft  to  treat  according  to 
pre-defined  criteria.  In  the  currently  available  version,  the  pair  of  aircraft  which  will  first 
lose  separation  is  selected,  then,  within  that  pair: 

•  The  aircraft  involved  in  most  conflicts 

•  The  aircraft  that  not  at  the  level  it  should  leave  at 

•  The  aircraft  that  has  furthest  to  go  before  leaving  -  because  this  is  easiest  to  return  to 
track  after  a  maneuver. 


Mr.  ■  ' 

1  q  =  QUIT 

■  • 

1  ; Uaiting ... 

0  0  0  "ffSTdf 


Controls 
T>Tine  (min) 
S>Speed  (20kt) 
F>FL  (Z0Fls) 
H>Hdcf  (10deg) 
R>Res  Nor  Nau 

+  =  Add  1 
-  =  Back  1 

X  =  Cancel 
?  =  try  out 
y  =  ACCEPT 


Figure  6  -  Aircraft  Trajectory 

Having  selected  an  aircraft,  the  system  shows  its  future  trajectory  superimposed  on  the 
display,  with  the  trajectories  of  conflicting  aircraft,  and  their  relative  positions  at  closest 
approach.  An  auxiliary  display  shows  the  flight  profile  of  the  aircraft,  with  a  horizontal 
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scale  representing  distance  to  the  same  scale  as  the  EDD,  and  a  vertical  scale  showing 
the  height  (flight  levels)  available.  Solid  blocks  represent  aircraft  which  will  be  in 
conflict,  and  hollow  blocks  represent  aircraft  which  pass  over  or  under  the  flight,  and 
therefore  would  be  in  conflict  if  the  flight  level  is  changed.  (Figure  6) 

The  controller  can  construct  an  order  to  solve  the  problems  of  this  aircraft.  (In  the 
demonstration  version  DEMFAST,  available  with  David  (1997),  keystrokes  are  used, 
because  the  software  available  does  not  support  a  mouse,  although  a  mouse  would  be 
preferable.)  As  the  order  is  constructed,  the  trajectory  is  revised,  so  that  the  consequences 
of  each  maneuver  can  be  seen.  The  controller  can  instruct  the  aircraft  to  return  to  track 
after  the  problems  have  been  solved.  Once  an  acceptable  order  has  been  developed,  the 
controller  accepts  it,  and  the  system  transmits  it  by  data-link  to  the  FMS  of  the  aircraft, 
adding  the  call- sign.  The  message  would  be  spoken  to  the  controller  and  to  the  pilot  and 
co-pilot  of  the  aircraft  addressed  (and  of  any  other  aircraft  in  the  vicinity)  by  voice 
synthesizers  in  the  language  of  their  choice,  eliminating  the  classic  Zagreb  problem 
(Weston  and  Hxirst,  1982),  The  controller  can  now  turn  his  attention  to  the  next  problem 
without  needing  to  verify  the  correct  reception  of  the  order,  or  to  remember  to  return  the 
aircraft  to  track  later. 

After  a  little  practice,  this  system  is  considerably  faster  than  spoken  control.  A  non- 
controller  can  handle  the  demonstrator’s  limit  of  40  aircraft  simultaneously,  where  each 
aircraft  remains  in  the  airspace  for  12  minutes,  corresponding  to  an  entiy  rate  of  200 
aircraft  per  hour,  at  twice  real-time  rate. 

In  a  real  system,  this  demonstrator  would  be  expanded  to  include  methods  of  monitoring 
deviations  from  track,  or  learning  more  about  the  aircraft.  Controllers  would  learn  to 
recognize  particular  ‘hot-spots’  in  the  traffic,  and  anticipate  problems,  as  they  do  in  the 
existing  system.  Additional  displays  might  be  designed  to  help  the  controllers  cope  with 
‘foreseeable  emergencies’,  such  as  emergency  decompression  or  engine  failure,  where 
one  controller  would  take  charge  of  the  aircraft  in  difficulties,  while  his  partner  would 
supervise  the  re-routing  of  other  aircraft. 

Back-up 

Although  controllers  enjoy,  and  are  good  at,  solving  conflicts,  there  may  be  occasions 
where  a  temporary  overload  means  they  cannot  solve  all  the  problems  in  time.  There 
should  be  a  process  triggered  by  the  equivalent  of  a  ‘short-term  conflict  alert’  to  ensure 
safety  if  potential  conflicts  are  not  solved  within  a  safety  limit.  The  ACAS  (Airborne 
Conflict  Avoidance  System)  is  in  fact,  such  a  system.  The  fail-back  system  should  be 
essentially  safe,  while  not  as  efficient  as  the  human  controller.  Given  the  nature  of  human 
beings,  the  temptation  to  rely  on  the  back-up  system  should  not  be  offered. 

Organizational  Repercussions 

The  alternative  display  system  designed  here  is  not  simply  a  cosmetic  re-design  of  the 
existing  system.  It  produces  a  considerable  increase  in  capacity,  reducing  the  number  of 
sectors,  which  itself  reduces  the  amount  of  coordination  involved.  It  reduces  the  mental 
workload  of  the  controller  at  the  cost  of  his  ability  to  maintain  a  continuous  ‘picture’  of 
the  present  and  future  traffic.  It  can  be  argued,  although  it  would  be  hard  to  prove,  that 
he  could  not  maintain  a  picture  of  this  amount  of  traffic  in  direct  flight  in  any  case.  The 


74 


traditional  distinction  between  the  Planning  and  the  Executive  controller  disappears.  (In 
real  Ufe  ATC  practice,  the  ‘official’  distinction  between  these  two  tasks  is  already 
considerably  blurred.)  The  ‘style’  of  control  changes  from  a  continuous  ‘hands-on’  style 
to  a  ‘control  by  exception’,  where  the  controller  must  react  to  the  (future)  problems 
detected  by  the  system,  and  the  immediate  problems  presented  by  emergencies.  This 
would  have  considerable  implications  for  initial  and  continued  training.  There  are  very 
good  reasons  not  to  reduce  the  control  team  to  one  man,  as  there  are  not  to  reduce 
aircrew,  (Yerkes  Law  -  “One  chimpanzee  is  not  a  chimpanzee  at  all”  (Yerkes  1945)  also 
applies  to  controllers)  As  for  aircraft,  the  trend  will  be  to  a  pair  of  controllers,  each 
capable  of  all  the  control  tasks.  Training  may  include  drills  for  the  allocation  of  tasks 
between  operators  (and  the  system),  as  with  CRM  (Cockpit  Resource  Management)  in 
civil  aviation. 

Concurrence 

Finally,  it  is  an  illusion,  induced  by  the  linear  nature  of  written  communication,  to  see 
these  stages  as  separate  steps  in  a  process.  In  reality,  each  step  will  interact  with  every 
other.  To  attempt  to  develop  a  complete  system  by  following  each  step  rigidly  is  rather 
like  trying  to  teU  a  car  driver  how  to  drive  from  one  place  to  another  by  providing  a 
detailed,  carefully  timed  set  of  instructions  how  to  move  the  steering  wheel,  the  clutch 
and  the  accelerator  for  an  hour  or  so.  The  processes  of  task  design  and  interface 
specification  are  inherently  interdependent.  Practically  this  means  that  crude  models 
rapidly  available  are  more  valuable  than  elegant  models  that  take  six  months  to  modify.  It 
implies  that  the  number  of  people  involved  in  the  design  and  development  phase  should 
be  as  small  as  possible.  (Ideally,  one.)  This  type  of  development  requires  the  attitudes 
and  commitment  described  in  ''Skunk  Works"'  (Rich  and  Janos,  1994),  although  the 
physical  resources  need  be  no  greater  than  a  standard  PC  with  a  good  display  screen. 

Conclusions 

Most  large-scale  real-time  control  systems  have  evolved,  rather  than  been  designed. 

The  advent  of  fast,  cheap  and  reliable  computer-based  information  transfer  and  analysis 
systems  presents  an  unprecedented  opportunity  to  re-design  such  systems  to  provide  a 
major  improvement  in  the  quality,  reliability  and  efficiency  of  the  system. 

It  is  not  sufficient  to  imitate  the  existing  display  system  on  a  CRT  screen.  The  actual 
control  task  should  be  re-analyzed,  and  the  human  operator  provided  with  a  satisfying 
task  appropriate  to  his  natural  abilities.  Tasks  the  human  fails  to  do  should  be  taken  over 
by  the  system,  although  it  is  important  that  the  system  should  be  safe,  rather  than  as 
efficient  as  the  human  operator. 

These  changes  imply  shifts  in  the  location  of  information  storage,  in  the  means  of 
transmission,  and  in  the  manner  in  which  operators  work. 
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Abstract 

Through  ongoing  progress  in  computer  hardware,  numerous  improvements  became  pos¬ 
sible  in  flight  simulation.  In  general,  low-cost  desktop  simulators  now  provide  possibili¬ 
ties  that  in  the  past  could  only  be  achieved  on  expensive  and  complex  systems.  In  addi¬ 
tion  to  a  quality  improvement  in  low-cost  desktop  simulation,  there  is  also  a  quantitative 
advantage.  A  wide  range  of  complex  tasks  can  nowadays  be  trained  on  relatively  simple 
and  inexpensive  equipment,  hi  fact,  a  number  of  tasks  can  already  be  trained  with  ’State 
of  the  Art  PC’s,  providing  the  opportunity  to  procure  a  larger  number  of  this  type  of 
simulator. 

This  paper  examines  whether  PC-based  equipment  can  be  used  to  support  training  of 
perceptual-motor  skills,  or  more  specific  manual  flying  skills,  i.e.  skills  that  to  date  sup¬ 
posedly  can  only  be  acquired  in  more  expensive  simulators  or  the  real  aircraft. 

The  motivation  for  the  research  described  in  this  paper  is  lead  by  the  basic  question: 
what  task-critical  features  are  needed  in  simulation  to  achieve  positive  transfer  to  manual 
flying  skills  in  real  flight? 

To  examine  this  question,  a  training  experiment  has  been  conducted.  The  objective  of 
this  experiment  was  to  test  the  training-effectiveness  of  PC -based  simulations  used  in 
advanced  manoeuvre  flight  training.  The  experiment  was  set  up  such  that  transfer-of- 
training  from  two  PC-based  training  devices-  a  standard  (Commercial  Off-  The-Shelf, 
COTS)  and  a  customized  PC-based  configuration  -  to  the  real  aircraft  could  be  exam¬ 
ined. 

Three  groups  of  trainee-pilots  participated  in  the  experiment.  Each  group  went  through 
identical  in-flight  training  to  learn  to  fly  a  complex  sequence  of  aerobatics  manoeuvres 
on  a  single-engine  propeller  aerobatics  aircraft.  The  second  and  third  group,  though,  re¬ 
ceived  extra  training:  each  in-flight  lesson  was  preceded  by  a  simulation  session  using 
the  respective  PC-based  configurations. 

It  was  found  that  the  group  trained  with  customized  PC-based  simulation  showed  a  clear 
advantage  over  the  two  other  groups,  trained  with  respectively  standard  PC-based  simu¬ 
lation  and  no  simulation.  The  use  of  customized  PC-based  simulation  leads  to  a  higher 
rate  of  learning  in  the  aircraft.  Most  remarkable  improvement  of  in-flight  performance  of 
the  group  trained  with  customized  PC-based  simulation  takes  place  in  the  first  lessons, 
giving  them  a  headstart,  causing  an  improvement  that  is  existent  throughout  the  remain¬ 
der  of  the  training.  It  is  discussed  that  this  higher  rate  of  learning  observed  by  trainees 
using  customized  simulation  can  be  accounted  for  by  the  following  abilities  learnt  in  the 
simulation  that  transfer  to  real  flight:  (1)  the  improved  ability  to  extract  visual  cues  from 
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the  outside  world  that  are  needed  to  perform  the  time-critical  elements  of  aerobatics  (2) 
speeded  up  learning  of  procedural  elements  of  aerobatics  in  an  earlier  stage  of  training 
(3)  an  improved  basis  for  in-flight  development  of  “seat-of-the-pants  skills”. 

Introduction 

This  research  applies  in  the  first  place  to  the  training  of  perceptual-motor  skills,  more 
specific  advanced  manual  flying  skills  for  military  operations.  Military  operations  with 
jets  and  helicopters  continuously  depend  on  perceptual  motor-skills.  A  real-life  military 
scenario  is  likely  to  be  a  continuous  sequence  of  very  complex  manoeuvres.  Additional 
tasks,  most  often  with  cognitive  and  procedural  aspects,  must  be  performed  in  parallel 
with  manoeuvering.  Therefore,  a  high  degree  of  skill  automatization  is  desirable  for  suc- 
cesful  completion  of  such  a  scenario.  Such  a  high  degree  of  automatization  of  perceptual- 
motor  skills  can  only  be  obtained  through  extensive  training.  Unfortunately,  shrinking 
military  budgets  and  increasing  public  environmental  awareness  restrict  the  number  of 
hours  that  military  pilots  can  spend  training  advanced  manoeuvres  on  real  aircraft. 

PC-based  systems  have  already  proved  useful  in  training  of  aircraft  system  knowledge, 
flight  management  system  programming,  radio-telephony  phraseology,  navigation,  and 
many  more  areas.  Thus  far,  such  systems  have  not  been  used  for  teaching  manual  flying 
skills.  These  skills  were  usually  taught  on  real  aircraft  or  on  training  devices  for  which 
the  design  was  most  often  guided  by  the  desire  for  high  engineering  fidelity  under  the 
assumption  that  the  more  a  training  device  is  like  the  real  aircraft,  the  better  will  be  the 
training. 

Above  restricting  circumstances  and  advances  in  PC-based  software  and  technology  pro¬ 
vide  a  reason  for  the  aviation  community  to  investigate  cost-effective  use  of  advanced 
PC’s  in  part-task  trainers  for  manual  flying  skills.  Not  with  the  intention  to  replace  the 
real  aircraft  training  or  the  full  flight  simulator  but  with  the  intention  to  support  the 
training  program,  thus  to  use  the  real  aircraft  and  the  full  flight  simulator  more  effec¬ 
tively  and  more  selectively. 

In  our  view,  for  a  part  of  flight  training,  the  term  low-fidelity  does  not  necessarily  imply  a 
sub-optimal  training  environment  when  compared  to  the  real  aircraft.  A  first  condition  for  an 
effective  low-fidelity  device  is  the  presence  of  identical  elements  that  are  critical  to  required 
task-performance,  allowing  the  trainee  pilot  to  develop  the  proper  response  strategy.  This 
characteristic  of  the  training  device  to  allow  the  development  of  a  proper  response  is  more 
important  than  having  an  exact  replica  of  the  cues  involved  (the  stimulus). 

Secondly,  the  low-fidelity  training  device  may  provide  some  instructional  strategies  superior 
to  those  attainable  in  the  real  aircraft.  Examples  of  such  instmctional  strategies  are  part-task 
and  adaptive  training,  extensive  feedback,  for  example  through  Knowledge  of  Results  (KR) 
followed  by  an  explanation  and  demonstration  of  the  correct  response,  provision  of  replay 
possibilities  and  cue  augmentation,  that  is  the  use  of  cues  that  are  not  present  or  have  insuf¬ 
ficient  cueing  effect  in  real  flight,  but  may  have  a  positive  effect  when  they  are  introduced  or 
amplified  in  the  training  environment.  Lintem  (1996)  provides  an  overview  of  transfer- 
theories  and  existing  research  on  flight  simulation  fidelity  research  to  date. 

The  current  research  is  distinguishable  from  earlier  research  in  the  field  of  training  effective¬ 
ness  of  low-cost/low-fidelity  simulation  because  of  its  focus  on  the  acquisition  of  perceptual 
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motor  skills  (aerobatics  skills),  in-flight  measurement  of  both  learning  curves  and  final  per¬ 
formance,  the  optimal  configuration  for  PC-based  simulation  and  the  use  of  augmented 
cues,  and  implications  for  selection  of  pilots. 

Method 

Experimental  set-up  and  hypotheses 

The  experiment  entails  a  comparison  between  three  groups  of  subjects  during  10  flying 
lessons;  the  first  8  flying  lessons  were  considered  as  training  and  the  last  2  flying  lessons 
considered  as  the  transfer  demonstration  phase.  The  experimental  set-up  involving  those 
three  groups  is  depicted  in  table  1 .  Choices  concerning  duration  and  number  of  lessons 
were  based  on  experience  and  common  sense. 


Control  Group  (C) 

Exp.  Group  1  (XI) 

Exp.  Group  2  (X2) 

In-flight  training 

10  lessons  of  30  mins, 
each. 

id.  to  C-group. 

id.  to  C-group. 

Simulation  training 

no  simulation  sessions 

10  simulation  sessions  of 
50  mins.  each. 

10  simulation  sessions  of 
50  mins.  each. 

Simulation  configura¬ 
tion 

Table  1:  Experimenta 

n/a 

/  training  set-up 

PC-based  standard 

equipment. 

PC-based  customized 

equipment  +  augmented 
cues. 

From  table  1  it  can  inferred  that  the  experimental  manipulations  were: 

•  the  use  of  PC-based  standard  equipment  in  simulation  sessions  for  group  XI , 

•  the  use  of  PC-based  customized  equipment  and  augmented  cues  in  simulation  ses¬ 
sions  for  group  X2. 


The  average  score  (see  following  section  on 
performance  measurement  and  scoring)  of 
each  group  in  the  transfer-demonstration 
phase  will  be  mutually  compared  under  the 
hypotheses  that  the  average  score  of  the 
XI -group  is  higher  than  that  of  the  C-group 
and  that  the  average  score  of  the  X2-group 
is  higher  than  both  those  of  XI-  and  C- 
group. 

The  same  hypotheses  will  be  tested^  with 
respect  to  the  average  progress  per  lesson 
Further,  the  magnitude  of  the  effects  (if  any) 
and  those  effects  will 


(learning  speed)  during  the  training  phase, 
resulting  from  the  experimental  manipulations  wil  be  investigated 
be  related  to  the  shape  of  the  learning  curves  for  the  three  groups. 


^  The  Statistica  software  package  (StatSoft,  1994)  will  be  used  for  statistical  analysis.  Significant  results 
will  have  a  p-level  smaller  than  0.05. 
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Task 

The  goal  of  the  aerobatics  training  was  to  fly  five  aerobatics  manoeuvres  in  a  fixed  order 
continuous  sequence  on  an  aircraft  (see  section  ‘aircraft’).  Those  five  aerobatics  ma¬ 
noeuvres  are  the  Loop,  Slow  Roll,  Inverted  Flight,  Immelmann  and  Split-S.  Each  of  those 
manoeuvres  takes  a  skilled  pilot  approximately  20  seconds  to  complete. 

The  order  Loop-Slow-roll-Inverted-flight~Immelmann-Split-S  was  a  logical  choice:  The 
loop  and  the  slow-roll  form  the  basis  for  both  the  Immelmann  (which  is  a  half-loop-half- 
roll)  and  the  Split-S  (which  is  a  half-roll-half-loop).  The  Slow-roll  is  the  basis  for  in¬ 
verted  flight  (which  is  a  half-roll,  level  inverted  for  15  seconds,  and  again  a  half  roll). 
Moreover  the  Split-S  was  the  last  manoeuvre  in  the  sequence  since  it  is  physically  the 
most  demanding,  mainly  caused  by  the  transition  from  negative  to  positive  g-force,  and 
also  the  most  critical  manoeuvre  with  respect  to  loss  of  altitude. 

Performance  measurement  and  scoring 

The  manoeuvres  were  judged  by  the  aerobatics  instructors  (instructor  ratings)  on  a  re¬ 
stricted  number  of  criteria.  The  criteria  were  chosen  and  prioritized  in  agreement  with  the 
instructors  on  the  basis  of  importance  of  a  particular  criterion  for  the  manoeuvre  and  the 
ability  to  observe  whether  a  criterion  is  fulfilled.  For  example,  a  g-force  of  3.5  to  4  g, 
pulled  at  loop-entry,  is  very  important  for  succesful  completion  of  the  loop  and  can  be 
observed  by  both  instructor  and  trainee  and  is  therefore  a  suitable  criterion.  On  the  other 
hand,  during  a  slow-roll  manoeuvre  it  is  very  hard  for  the  instructor  to  observe  and  judge 
the  roll-rate  in  an  absolute  sense,  although  it  is  an  important  criterion.  Therefore  a  roll 
rate  of  say,  20  degrees  per  second,  is  not  a  suitable  criterion  as  a  basis  for  instructor  rat¬ 
ing.  In  this  case  a  relative  criterion  (e.g.  roll-in-rate  equals  roll- out-rate)  is  more  suitable. 

A  sufficient  small  number  of  criteria  were  chosen,  such  that  the  instructor,  seated  behind 
the  subject  on  the  back  seat  of  the  airplane,  was  able  to  fill  out  the  pre-printed  score- 
forms.  Scoring  was  based  on  criteria  of  equal  importance,  that  is  aU  criteria  had  an 
identical  weight  factor.  All  manoeuvres  were  considered  as  being  of  equal  importance, 
that  is  were  scored  on  an  equal  number  of  criteria.  During  the  in-flight  lessons,  the 
trainee  gained  a  point  for  each  criterion  that  was  fullfilled,  that  is  when  the  associated 
parameter  value  fell  within  the  acceptable  range  (see  table  2  for  an  example). 


Para¬ 

meter 

Phase  in  manoeuvre 

Ideal  value 

^^acceptable^^ 

range 

Roll  an¬ 
gle 

pull-up  (phase  4) 

0  deg.  Constant 

-5  tot  +5  deg. 

Max  g 

pull-up  (phase  4) 

4g  at  entry  speed  of  145  mph 

3.5  to  4g 

Roll  an¬ 
gle 

top  (phase  7) 

0  deg.  Constant 

-10  to  -flO  deg. 

Heading 

top  (phase  7) 

Constant 

-5  to  4-5  deg. 

Altitude 

exit  loop  (phase  10) 

-  100  to  -1-100  feet 

Table  2:  Example:  criteria  for  scoring  the  quality  of  the  loop 


Selection  of  subjects  for  the  experiment 

In  order  to  compose  three  groups  of  8  subjects,  24  subjects  were  needed  for  the  experi¬ 
ment.  These  were  selected  in  three  selection-rounds. 
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In  co-operation  with  a  school  for  commercial  pilots,  approximately  60  trainee  pilots  at¬ 
tended  a  briefing  and  completed  registration  forms.  The  first  selection-round  took  place 
on  the  basis  of  body  weight  (max.  80  kg,  because  of  the  restriction  in  take-off  weight  of 
the  airplane),  age  (max.  27  yrs),  fixed-wing  flying  experience  (max.  250  hrs),  previous 
aerobatics  experience  (no  previous  aerobatics  experience  allowed). 

With  a  remaining  number  of  31  trainee  pilots,  in  a  second  selection  round,  the  Aiming 
Screening  Task  (AST)  was  used  to  collect  data  on  individual  ability.  The  AST  was  de¬ 
veloped  for  the  Learning  Strategies  Program  (Donchin  et  al,  1989).  This  eye-hand  co¬ 
ordination  task  was  used  since  it  is  a  well  know  selection  task  for  research  into  percep¬ 
tual-motor  skills.  The  objective  of  the  AST  is  to  destroy  as  many  targets  on  a  computer- 
screen  as  possible  within  a  session  of  2  minutes,  using  a  joystick.  The  task  consists  of 
three  sessions  of  which  the  highest  score  (number  of  targets  destroyed)  was  taken  as  the 
selection  measure. 

In  the  current  research  the  AST  was  not  used  to  remove  potential  subjects  from  the  actual 
experiment  but  merely  for  balancing  purposes  between  the  three  groups  (conditions).  On 

the  basis  of  AST  score,  4 
ability-levels  were  distin¬ 
guished:  Low  (L),  Low- 
Medium  (LM),  High- 
Medium  (HM)  and  High  (H), 
corresponding  with  respec¬ 
tive  quartiles  of  the  distribu¬ 
tion  depicted  in  figure  2. 
After  assignment  of  subjects 
to  groups,  each  group  of  8 
subjects  contained  2  L,  2 
LM,  2  HM  and  2  H  subjects. 
The  remaining  subjects  were 
assigned  to  a  spare  group. 
The  third  selection  round 
consisted  of  a  20  minutes  test-flight  on  a  single-engine  propeller  aircraft  of  make  Fuji,  a 
side-by-side  seater  with  aerobatics  capabilities.  Deliberately,  this  aircraft  was  of  a  differ¬ 
ent  type  than  was  used  for  the  actual  experimental  training  to  avoid  confusion  between 
the  pre-training  and  training-phase.  The  subject  had  to  perform  a  number  of  basic  flight 
manoeuvres  and  a  number  of  aerobatics  manoeuvres,  the  latter  deliberately  being  differ¬ 
ent  than  those  included  in  the  actual  experimental  training.  The  test-flight  had  two  ob¬ 
jectives:  firstly  to  collect  data  on  the  basic  and  advanced  flying  abilities  of  the  subject 
and  secondly  to  test  and  possibly  remove  subjects  that  had  insufficient  physical  resis- 
tence  against  air  sickness  to  complete  the  aerobatics  training,  A  pre-printed  score-form 
was  used  by  the  instructor.  On  the  basis  of  test-flight  results  three  subjects  were  removed 
from  the  experiment  and  replaced  by  subjects  from  the  spare  group.  To  avoid  a  potential 
bias  in  the  behaviour  of  subjects,  due  to  knowledge  of  the  objectives  of  the  experiment, 
the  experimenters  presented  the  project  as  a  study  of  in-flight  instructional  strategies 
rather  than  a  study  into  PC-based  training.  The  three  groups  were  trained  separately. 


.  Expected 

300  400  500  600  700  800  900  1000  1100  1200  Normal 

Aiming  Screening  Task  scores 

Figure  2:  Distribution  of  Aiming  Screening  Task  Scores 
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Training  procedure 

Table  3  represents  the  total  training  structure  for  the  different  groups.  The  contents  of 
the  lessons  in  each  phase  will  be  briefly  described,  starting  with  the  training  elements 
that  all  groups  had  in  common. 

In  the  pre-training  phase,  all  subjects  received  the  training  manual.  The  theory  in  the 
manual  had  to  be  mastered  before  the  start  of  the  training.  Before  entering  the  flight 
training,  the  subjects  had  to  go  through  a  theoretical  test  on  the  contents  of  the  manual. 


c 

Theory  and  test 

Flight  lesson  1 

Flight  lesson  2 

Flight  lesson  3-8 

Flight  lesson  9 

Flight  lesson  10 

XI 

X2 

Theory  and  test 

Sim. 

Session 

2 

B 

1 

B 

Pre¬ 

training 

Fam.  Phase 

Training  phase 

Transfer-demonstration  phase 

Table  3:  Organization  of  training  sessions  for  C  group  and  XI /X2  groups 


Before  and  after  each  flight  lesson  the  trainee  received  a  briefing  by  the  instructor.  The 
first  flight  lesson  (Familiarization  phase,  see  table  3)  was  used  to  familiarize  with  the 
aircraft.  Furthermore,  the  instructor  demonstrated  the  complete  sequence  of  five  ma¬ 
noeuvres  after  which  the  trainee  could  try  the  sequence  him/her  self  in  a  talk-through 
manner. 


Figure  3:  Organization  of  a  flight  lesson  in  the  training  phase 

In  the  training  phase  (flight  lesson  2-8)  a  part-task  instruction  strategy  (see  Wightman 
and  Lintem,  1985)  was  employed.  The  basic  organization  of  each  in-flight  lesson  is  de¬ 
picted  in  the  flowchart  of  figure  3.  A  logical  choice  was  to  segment  the  whole  task  (fly- 
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ing  the  sequence)  into  its  component  manoeuvres  1-5  (loop  =1,  slow-roll  =  2,  etc.)  and 
train  those  as  part-tasks.  Thus,  the  second  in-flight  lesson  started  with  trials  on  the  least 
complicated  manoeuvre  (the  loop).  If  this  manoeuvre  could  be  flown  according  to  the 
criteria,  trials  on  the  next  manoeuvre  (the  slow-roll)  were  undertaken  and  so  on  .  Once  all 
separate  manoeuvres  were  mastered,  the  same  order  of  manoeuvres  was  repeated  (outer 
loop  of  figure  3). 

In  the  transfer  demonstration  phase  (lesson  9-10,  see  table  3),  the  trainee  had  to  demon¬ 
strate  the  whole  sequence  of  five  manoeuvres  in  a  continuous  and  errorless  fashion.  Em¬ 
phasis  was  on  accuracy:  “try  to  fly  the  sequence  as  accurate  as  possible”. 

Both  XI  and  X2  group  subjects  received  50  minutes  simulation  sessions  preceding  each 
in-flight  lesson.  Simulation  sessions  for  both  groups  were  identically  organized. 

The  first  simulation  session  (in  the  familiarization  phase,  see  table  3)  consisted  of  fa¬ 
miliarization  with  the  simulation,  a  trial  on  each  manoeuvre  and  a  trial  on  the  complete 
sequence. 

Simulation  sessions  2-10  consisted  of  (1)  structured  repetition  of  problematic  manoeu¬ 
vres  as  indicated  by  the  flight  instructor  during  the  previous  in-flight  lesson  and  (2) 
preparation  of  new  manoeuvres.  No  mediation  by  instructors  was  needed. 

Aircraft 

The  aerobatics  aircraft  used  for  the  train¬ 
ing  is  of  type  Bellanca  Super  Decathlon 
depicted  in  Figure  4.  This  single-engine 
propeller  aircraft  is  thrusted  by  a  four 
cylinder  180  HP  Lycoming  piston  engine. 

The  aircraft  can  take  g-loads  in  the  range  - 
5  to  6  G.  Equipment  on-board  of  the  air¬ 
craft  was  used  for  recording  12  relevant 
flight  parameters  during  the  training.  The 
recorded  data  is  not  yet  taken  into  account 
in  this  article. 


Figure  4:  The  Bellanca  Super  Decathlon 


^  For  didactical  reasons,  the  instructor  had  the  possibility  to  start  with  a  next  manoeuvre 
even  though  not  all  criteria  were  fulfilled.  The  rate  with  which  trials  were  repeated  was 
left  to  the  decision  of  the  instructor  and  his  opinion  on  the  skill-level  and  prospects  of  the 
trainee. 
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PC‘based  training  equipment  and  software 

Both  groups  used  the  aerobatics  training  software  package  “Flight  Unlimited”  from 
Looking  Glass  Technologies  (MS-DOS  version.  Looking  Glass  Technologies,  1995). 

Flight  Unlimited  is  a 
PC  flight  simulation 
with  relatively  accu¬ 
rate  aircraft  flight 
models  (including 
that  of  the  aircraft 
used  in  the  experi¬ 
ment)  and  3D 
photo-realistic  land- 

FigureS:  Controls  (throttle,  pedals  and  stick)  for  the  XI -group  scapes.  A  view  was 

selected  in  which  on  the  lower  half  of  the  screen  the  instruments  were  depicted,  while  the 
upper  half  of  the  screen  was  used  for  out-of-the- window  view.  Furthermore  the  “ren¬ 
dered’  was  set  up  such  that  terrain  was  in  lowest  detail,  haze  was  on  highest  level,  clouds 
were  turned  off,  blending  set  on  lowest  level,  g-effects  were  on  and  wind  strength  was  set 
to  a  maximum. 


XI-  and  X2-groups  used  the  same  simulation  software  and  the  same  PC  (Pentium  166 
equipped  with  a  sound  card  and  speakers  and  a  CD-ROM  player).  The  differences  in 
equipment  are  listed  in  table  4. 


XI -group 

X2-group 

PC-monitor 

17”  colour  monitor, 
used  in  VGA-mode 
(640x480  pixels) 

21”,  further  identical  to  XI 

Throttle 

Pro  Throttle  by  CH- 
products  (see  figure  5) 

Throttle,  rudder  pedals  and  stick  were  custom¬ 
ized  and  mounted  on  a  base  such  that  their 
shape,  position  and  displacement  resembled 
those  in  the  aircraft  (see  figure  6) 

Pedals 

Pro  Pedals  by  CH- 
products  (see  figure  5) 

Stick 

Flightstick  Pro  by  CH- 
products  (see  figure  5) 

Seating 

Standard  bureau  chair 

Mounted  on  base  and  resembled  the  chair  in  the 
aircraft  such  that  trainees  were  able  to  occupy 
the  same  bodily  posture  as  in  the  aircraft  (see 
figure  6) 

Table  4:  differences  in  equipment  configurations  between  XI-  and  X2-groups 


Furthermore,  subjects  of  the  X2-group  were  presented  with  the  following  augmented 
cues  during  their  simulation  sessions: 
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•  guidance  by  a  digitized  instructor  voice  (feature  of  the  Flight  Unlimited  software)  during 
their  simulation  sessions  (subjects  were  “talked  through”  the  manoeuvre); 

•  lesson  arrows  (feature  of  the  Flight  Unlimited  software),  those  are  indications  for 
suggested  stick  and  pedal  movements  were  presented  in  real-time  on  the  screen 

Results 

Invalid  data  and  outliers 

For  methodological  reasons,  data  from  three  subjects  had  to  be  removed  from  the  ex¬ 
periment.  From  the  C-group,  data  from  one  subject  was  removed  from  the  analysis  since 
incorrect  information  had  been  received  from  the  flying  school.  On  the  basis  of  new  in¬ 
formation,  it  was  decided  that  this  subject  could  not  be  considered  as  representative  for 
the  trainee  pilot  population  under  study.  From  the  XI -group,  data  from  one  subject  was 
removed  since  this  subject  had  a  physical  problem  that  hindered  the  subject  to  look 
sidewards  under  the  high  g-forces  that  occurred  in  most  manoeuvres,  and  could  therefore 
not  complete  the  aerobatics  training.  From  the  X2-group,  data  from  one  subject  had  to  be 
removed  since  this  subject  turned  out  to  have  significantly  more  fixed  wing  experience 
than  initially  reported.  Furthermore,  one  outlier  (unusual  low- score  score  of  a  C-group 

subject)  was  removed  on  the  basis  of  the 
3s-criterion  used  in  residual  analysis. 

Instructor  ratings 

It  must  be  noted  that  the  results  currently 
presented  in  this  article  are  based  on  in¬ 
structor  ratings.  ^  The  three  instructors 
were  randomly  assigned  over  a  total  of 
240  flight-lessons  and  were  instructed  to 
score  using  pre-printed  score  forms  with 
performance  criteria  in  terms  of  objective 
and  observable  flight  parameters  (see  ex¬ 
ample  in  table  2).  To  examine  inter-rater 
reliability,  stepwise  regression  analysis 
was  conducted,  in  which  the  variables  rep- 
Figure  6:  Customized  PC-based  trainer  for  resenting  the  instructors  (3  lev- 
the  X2-group  els/instructors  coded  in  two  dummy  vari¬ 

ables)  were  two  of  several  independent 
variables  (simulation-configuration,  ability-scores  measured  in  the  pre-training  phase, 
etc.)  and  in  which  the  score  during  the  transfer-demonstration  phase  was  the  dependent 
variable.  The  dummy  variables  representing  the  instructors  did  not  significantly  contrib¬ 
ute  to  the  model  for  regression,  nor  did  those  variables  account  for  a  substantial  percent¬ 
age  of  the  variance  in  scores.  It  is  realised  that  these  results  do  not  formally  prove  inter¬ 
rater  reliability.  In  view  of  time  constraints  and  the  availability  of  objective  measure¬ 
ments  more  formal  tests  have  been  abandoned  for  this  article. 


^  At  the  time  of  preparing  this  article  the  analysis  of  the  recordings  with  the  airborne 
measurement  equipment  is  still  ongoing  and  will  be  reported  in  the  course  of  1997. 
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Representation  of  in-flight  scores 

The  instructional  strategy  employed  in  the  training-phase  was  directed  on  leaming-by- 
doing  and  thereby  flying  as  many  manoeuvres  as  possible.  A  different  approach  was 
taken  in  the  transfer-demonstration  phase  where  emphasis  was  put  on  demonstrating  a 
restricted  number  of  sequences  wiA  the  highest  possible  accuracy.  For  the  training 
phase,  we  therefore  calculated  the  in-flight  score  per  lesson  by  summing  the  total  number 
of  points  gained  in  that  lesson,  thus  representing  both  speed  (number  of  trials  per  lesson) 
and  accuracy  of  the  manoeuvres.  For  the  transfer-demonstration  phase  however,  we  will 

calculate  the  in  flight  score  by  di¬ 
viding  the  points  gained  in  flying  the 
sequence  by  the  number  of  se¬ 
quences  flown  in  that  phase,  thus 
merely  representing  the  accuracy  of 
the  sequence.  For  interpretation  and 
comparison  purposes,  all  scores  will 
be  scaled  to  100%,  through  division 
by  maximum  attainable  accuracy 
score  and  maximum  number  of  ma¬ 
noeuvres  per  lesson  observed. 

Transfer-demonstration 
The  objective  of  the  transfer- 
demonstration  phase  was  to  fly  con¬ 
tinuous  sequences  with  maximum 
possible  accuracy.  Mean  scores  for  the  three  groups  are  depicted  in  figure  7.  The  C- 
group  scored  89.3%  (M=89,3,  s=13.6),  the  Xl-group  scored  92.5%  (M=:92,5,  s=5,5)  and 
the  X2-group  scored  98.2%  (M=98.2  s=2.91).  Wilcoxon  matched  pairs  test  was  used  to 
compare  means.  The  difference  in  mean  score  between  C  and  XI  group  is  not  significant, 
the  difference  between  C  and  X2  group  is  significant  (p=0.017),  the  difference  between 
XI  and  X2  group  is  significant  (p=0.015). 

Thus,  while  the  Xl-group,  that  is  the  group  using  the  standard  simulation  equipment, 
failed  to  show  significant  transfer,  the  X2-group  proved  to  have  a  significant  advantage 
from  using  the  customized  configuration.  The  differences  in  performance  score  in  the 
transfer  demonstration  phase  are  relatively  small  (The  X2-group  scored  only  approx.  9% 
better  that  the  C-group).  However,  it  must  be  noted,  that  scores  had  a  tendency  towards 
100%  in  the  transfer-demonstration  phase.  Since  scores  were  based  on  a  restricted  num¬ 
ber  of  relatively  simple  criteria,  near-maximum  accuracy  could  relatively  easy  be 
achieved.  This  ceiling-effect  in  performance  score  (based  on  accuracy)  is  thought  to 
mask  the  real  difference  in  skill  level  in  the  transfer  demonstration  phase.  Further  analy¬ 
sis  of  in-flight  data  on  the  basis  of  a  larger  number  of  more  sophisticated  scoring  criteria 
would  reveal  whether  this  assumption  is  true. 


Figure  7:  Mean  scores  in  transfer  demo  phase 
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Learning  speed 


The  in-flight  learning 
curves  for  the  three 
groups  are  depicted  in 
figure  8.  Learning 
speed  is  evaluated  in 
terms  of  average 
score-improvement 
per  lesson.  It  can  be 
seen  from  figure  8 
that  initial  levels  of 
performance  (lesson 
1)  are  approximately 
equal.  Looking  at  the 
training  phase  as  a 
whole,  the  C-group 
had  an  average  im- 
Figure  8:  Learning  curves  provement  of  4.6% 

per  lesson,  the  XI- 

group  had  an  average  improvement  of  5.3%  per  lesson  and  the  X2-group  had  an  average 
improvement  of  6.9%  per  lesson.  The  differences  in  improvement  per  lesson  between  C- 
and  X2-group  and  XI-  and  X2-group  are  significant  (p  resp.  0.00045  and  0.0025),  while 
the  difference  between  C  -  and  XI -group  is  not  significant  (p=0,14).  From  figure  9  it 
can  be  seen  that  the  most  remarkable  improvement  of  the  X2-group  (approx.  20%)  is 
booked  early  in  the  training  phase  and  is  existent  throughout  the  remainder  of  the  train¬ 
ing. 

Linear  approximation  of  learning  curves 

In  order  to  assess  aerobatics  skill-level  during  the  training  phase  of  the  three  groups  in 
terms  of  speed  and  accuracy  under  the  assumption  that  performance  wiU  eventually 
level-off  at  an  asymptotic  level  the  scores  per  lesson  were  approximated  by  linear  curves, 
using  the  same  data  as  in  figure  8  (see  figure  9),  Results  of  the  analysis,  using  least 
square  approximation  are  included  in  table  5. 


Model:  Score  =  A  *  lesson  no.  +  B 

Group 

Intercept  B 
(begin  level) 

Slope  A  (learn¬ 
ing  speed) 

/F  (coefficient  of  de¬ 
termination) 

F -value  (overall 
model  utility) 

p-level 

C 

16.4  % 

4.6  %  per  lesson 

93% 

76 

<0.01 

XI 

15,7  % 

5.3  %  per  lesson 

99% 

418 

<0.01 

X2 

17.5  % 

6,9  %  per  lesson 

94% 

88 

<0.01 

Table  5:  results  of  linear  approximation  of  the  learning  curves 


Figure  9:  Linear  approximation  of  learning  curves 


These  results  show  that  all  learning  curves  are  approximately  linear  and  do  not  curve  to¬ 
wards  an  asymptote,  indicating  that  skill  level  could  be  further  improved  and  the  training 
phase  was  too  short  to  approach  the  maximum  attainable  skill  level  with  the  task  given. 

Ability  data  and  relation  with  aerobatics  performance 

It  was  hypothesized  that  ability  data  collected  in  the  pre-training  phase  (AST-scores, 
standard  flying  skills  scores  and  advanced  flying  skill  scores)  are  good  predictors  for 
aerobatics  performance.  To  evaluate  their  relative  predictive  value  ability  scores  were 
included  as  variables  in  a  standard  multiple  regression  analysis  together  with  the  inde¬ 
pendent  variable  of  tlie  experiment.  For  this  purpose,  the  experimental  manipulation  of 
the  experiment  at  three  levels  (no  simulation,  standard  simulation,  customized  simula¬ 
tion)  is  coded  with  two  qualititive  dummy  variables  XI  and  X2.  The  dependent  variable 
is  the  performance  score  (accuracy)  obtained  during  the  transfer  demonstration  phase. 
The  results  are  summarized  in  table  6  (significant  results  are  underlined). 
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Model: 

Score  (transfer  demonstration  phase)  =  Bo+Bi*AST+B2*STD+B3*ADV+B4*Xl+B5*X2 

R  =  0.64,  R^=41%,  F(5.35W.77.  p<0.002 _ _ _ 

Variable 

Magnitude  Bn  of  the  effect 
scaled  to  a  range  [-1,1] 

t(35) 

p-levcl 

Aimine  Screenine  Task  f  AST) 

0.28 

u 

0.042 

Standard  flying  task  (STD) 

-0.16 

-0.95 

0.347 

Advanced  flvine  task  (^ADV) 

0.485 

2.94 

0.005 

Use  of  COTS  simulation  (XI) 

0.078 

0.51 

0.611 

Use  of  customized  simulation  (X2) 

0.49 

3.17 

0.003 

Table  6:  Multiple  regression  analysis  results,  relating  several  variables  to  aerobatics 
performance  in  the  transfer  demonstration  phase. 


The  results  from  the  analysis  show  that  advanced  flying  ability  (ADV,  measured  on  three 
advanced  manoeuvres:  wing  over,  barrel  roll  and  spiral  dive),  use  of  customized  PC 
simulation  (X2)  and  AST-score  yield  significantly  to  the  prediction  of  aerobatics  per¬ 
formance  in  the  transfer  demonstration  phase,  of  which  the  latter  one  is  of  less  impor¬ 
tance,  in  terms  of  the  magnitude  of  the  effect.  The  use  of  standard  PC-simulation  does 
not  contribute  significantly  to  the  model,  as  could  be  expected  from  our  earlier  analysis. 
Standard  flying  ability,  as  measured  during  9  standard  manoeuvres,  does  not  seem  to 
bear  a  correlation  with  aerobatics  performance. 

Conclusions  and  discussion 

Research  has  been  carried  out  to  evaluate  transfer-of-  training  of  perceptual  motor  skills 
(aerobatics  skills)  to  the  real  aircraft  from  three  different  types  of  ground-training.  It  was 
found  that  the  group  that  was  trained  with  customized  PC-based  simulation  (X2)  showed 
a  clear  advantage  over  the  two  other  groups,  trained  with  respectively  standard  PC-based 
simulation  (XI )  and  no  simulation  (C). 

The  learning  curves  of  the  three  groups  show  that  those  results  apply  to  the  initial  skill 
acquisition  phase,  where  the  learning  curve  is  approximately  linear  and  skill-levels  are 
one  relatively  low  level  of  automaticity.  In  this  phase,  the  use  of  customized  PC-based 
simulation  leads  to  a  higher  rate  of  learning.  During  the  training  phase,  the  group  with 
no-simulation  (C)  was  able  to  improve  their  performance  by  approx.  37%,  the  group  with 
standard  simulation  (XI)  by  approx.  42%  and  the  group  with  customized  simulation 
(X2)  by  approx.  55%.  Most  remarkable  improvement  in  score  of  the  X2-group  takes 
place  in  the  first  lessons,  giving  them  a  headstart,  causing  an  improvement  that  is  exis¬ 
tent  throughout  the  remainder  of  the  training. 

Concerning  the  validity  of  these  conclusions,  the  question  may  be  raised  whether  the 
flight  performance  differences  between  the  X2-  and  respectively  the  XI-  and  C-group 
can  be  accounted  for  by  other  factors  than  the  imposed  experimental  manipulations.  A 
possibility  that  could  give  the  X2-group  an  advantage  over  the  XI-  and  C-group  and  not 
stenaming  from  the  customized  simulation  itself  is  biased  treatment  by  the  flight  instruc- 
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tors,  since  it  was  practically  impossible  the  keep  the  flight-instructors  unaware  of  the 
ground-training  that  each  group  received.  To  minimize  the  likelyhood  of  this  possibility, 
scoring  criteria  and  scoring  standards  were  used  that  left  the  least  room  for  free  interpre¬ 
tation  and  subjective  judgement.  Further  validation  of  the  results  with  the  performance 
data  logged  with  the  airborne  equipment  and  during  simulation  is  needed  to  fully  exclude 
this  possibility  and  to  gain  more  insight  in  the  reliability  of  instructor  ratings. 

Viewing  the  conclusions  of  this  research,  the  key  question  is  raised:  Why  does  the  X2 
group  show  significant  positive  transfer,  where  the  XI  group  does  not?  What  makes  the 
difference  in  training  effectiveness  of  perceptual  motor  skills  when  using  customized  PC- 
based  simulation? 

A  prominent  and  valued  improvement  (by  expert  aerobatics  pilots)  of  the  customized 
simulation  over  the  COTS  simulation  was  the  realistic  position  and  displacement  of 
stick,  rudder-pedals  and  thottle  relative  to  the  body  of  the  pilot.  Our  interpretation  of  the 
results  is  that  by  providing  a  realistic  design  of  the  workspace  in  simulation,  perceptual- 
motor  strategies  transfer  better  to  real  flight,  thus  decreasing  the  need  for  paying  (visual) 
attention  to  the  different  configuration  of  stick,  rudder-pedals  and  throttle  during  real 
flight.  The  observation  that  subjects  in  the  XI -group  noticeably  payed  visual  attention  to 
stick-handling  during  simulation- sessions  further  supports  this  view.  Also,  we  hypothe¬ 
size  on  the  basis  of  the  results  that  realistic  design  of  the  simulation  workspace,  such  that 
the  trainee  occupies  a  postural  position  that  is  identical  to  that  in  normal  (straight  and 
level)  flight,  serves  a  baseline  cueing  requirement  for  further  development  (in-flight)  of 
seat-of-the-pants  skills,  i.e.  skills  that  relate  to  the  perception  and  processing  of  vestibu¬ 
lar,  tactile  and  kinaesthetic  cues.  We  further  hypothesize  on  the  basis  of  the  results  that 
the  use  of  augmented  cues,  i.e.  a  digitized  instructor  voice  during  the  simulation  lessons 
and  the  use  of  the  lesson  arrows  (visual  indication  of  the  direction  in  which  to  move  the 
rudder  pedals  and  stick)  made  learning  of  the  procedural  components  of  the  manoeuvres 
easier,  thereby  decreasing  the  cognitive  load,  such  that  during  simulation  more  attention 
could  be  paid  to  the  visual  components  of  the  manoeuvres.  The  in-flight  observation  by 
instructors  that  X2-subjects  had  a  better  out-of- the- window  visual  orientation,  while  XI- 
subjects  seemed  to  concentrate  more  on  the  cockpit  instruments,  supports  this  view.  It  is 
thought  that  the  use  of  a  larger  monitor  (21”  rather  than  17”)  in  the  customized  simula¬ 
tion  has  further  contributed  to  positive  transfer.  The  partial  contributions  of  those  cus- 
tomizations  could  not  be  established  experimentally  and  is  worthwile  focussing  upon  in 
further  research. 

The  higher  rate  of  learning  observed  during  the  in-flight  lessons  with  the  group  that  used 
customized  simulation  can  thus  be  explained  by  (1)  their  improved  ability  to  extract  vis¬ 
ual  cues  from  the  outside  world  that  are  needed  to  perform  the  time-critical  elements  of 
aerobatics  (2)  speeded  up  learning  of  procedural  elements  of  aerobatics  in  earlier  stage 
in  training  (3)  an  improved  basis  for  in-flight  development  of  seat-of-the-pants  skills. 

An  additional  factor  that  is  hard  to  catch  in  figures  or  concrete  observations,  but  may 
well  be  overlooked,  is  a  possible  higher  level  of  motivation  by  the  X2-group,  i.e.  to  train 
with  the  customized  configuration  was  notably  more  fun  than  to  train  with  the  standard 
configuration.  Although  all  participating  subjects  were  highly  motivated,  differential  ex¬ 
trinsic  motivation  caused  by  the  differences  in  ground  training  may  have  added  to  incre- 
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mental  transfer  of  the  X2-group, 


This  research  has  contributed  to  the  fidelity  database  for  flight  simulation  by  focussing 
on  some  low-cost,  state-of-the-art,  multi-media  solutions.  Configurations  such  as  those 
experimentally  evaluated  can  be  applied  to  several  settings:  (1)  Customized  PC-based 
simulations  can  be  used  to  support  the  first  stage  of  in-flight  training  and  training  on 
high-fidelity  moving  base  simulators.  (2)  It  was  observed  in  the  current  training  program 
that  trainees  in  the  XI-  and  X2-groups  needed  very  little  pre-flight  briefing.  Briefing 
times  went  down  from  approx.  15  minutes  for  the  C-group  subjects  to  approx.  5  minutes 
for  the  XI -group  subjects  to  almost  zero  for  the  X2-group  subjects.  Thus,  both  simula¬ 
tion  configurations  could  be  applied  as  automatic  briefing  tools,  therewith  saving  flight- 
instructor  time.  (3)  It  is  a  well-known  fact  from  learning  psychology  that  lower-abihty 
subjects  benefit  more  from  advanced  instructional  strategies  than  higher-ability  subjects. 
Although  not  yet  apparent  from  the  current  results,  it  is  not  unthinkable  to  employ  cus¬ 
tomized  PC-based  simulation  as  a  remedial  instruction  tool  throughout  initial  training, 
therewith  lowering  wash-out  rates  in  this  stage  of  training.  (4)  The  data  in  this  experi¬ 
ment  reveals  that  performance  on  standard  flying  tasks  is  by  no  means  a  good  predictor 
for  aerobatics  performance,  probably  since  aerobatics  tasks  appeal  more  to  seat-of-the- 
pants  flying.  Further  research  could  therefore  be  directed  to  the  utility  of  customized  PC- 
based  simulations  for  the  selection  of  military  pilots. 
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Abstract 

For  its  central  theme  this  paper  will  address  some  of  the  problems  which  arise  because 
the  human  is  often  considered  as  the  weakest  link  in  the  aviation  system.  By  striving  for 
a  systematic  integrated  approach  of  the  technical  and  human  considerations  the  authors 
suggest  that  all  the  players  in  the  aviation  system  can  make  a  contribution  to  improving 
aviation  safety.  The  paper  addresses  a  number  of  human  factors  aspects  need  to  be 
addressed  in  detail  if  they  are  to  act  as  effective  drivers  in  the  efforts  to  improve  aviation 
safety. 

The  advent  of  modem  multimedia  techniques  suggests  that  a  paradigm  shift  in 
capabilities  has  now  occurred.  This  allows  us  to  realistically  model  complex  large-scale 
systems  whilst  also  allowing  individuals  to  specifically  address  their  particular  area  at 
whatever  level  of  complexity  is  appropriate  to  them.  The  paper  will  look  at  how 
multimedia  techniques  are  particularly  applicable  to  some  of  the  aviation  issues 
addressed  earlier  in  the  paper.  This  theme  will  be  central  to  establishing  the  benefits 
which  multimedia  techniques  can  bring  to  the  aviation  system  in  order  to  create  an 
effective  and  systematic  integrated  approach  to  the  human  performance  interactions  in 
the  cockpit  and  air  traffic  control. 

Introduction 

Human-system  interactions  have  recently  been  the  focus  of  extensive  study  and 
discussion.  A  central  point  of  this  effort  has  been  the  recognition  that  the  human  is 
often  the  weakest  link  in  the  system  framework.  This  is  particularly  true  within  the 
aviation  community.  Human  error  is  the  most  cited  single  cause  of  aviation  accidents. 
This  is  not  surprising  when  one  considers  the  myriad  of  aspects  and  stages  of  the  aviation 
system  development  and  implementation  where  the  human  is  a  major  component:  design 
and  manufacture,  operators,  maintainers,  managers  and  trainers. 

Human  Factors  in  aviation 

The  connections  among  some  of  the  various  aviation  activities  can  be  illustrated  by  the 
diagram  shown  in  fig.l. 
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FIG1. 


As  the  model  illustrates,  the  human  is  central  to  all  aspects  except  the  ambient  weather 
conditions.  The  challenge  is  to  address  and  optimize  the  interactions  among  all  of  these 
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humans  so  that  the  system  functions  as  effectively  and  efficiently  as  possible. 
Multimedia  is  often  touted  as  the  newest  and  most  effective  solution  to  all  of  the 
problems  that  plague  the  aviation  human  interactions.  However,  multimedia 
presentation  of  information  both  in  the  cockpit  and  on  the  ground  has  been  in  use  for 
decades. 

First  consider  the  cockpit.  Flight  information  is  displayed  on  color  LCD  primary  and 
secondary  displays  in  many  modem  aircraft.  The  identification  of  some  controls  is 
presented  as  labels  and  in  some  cases,  the  controls  are  identifiable  by  shape.  Digital 
AXIS  messages  are  present  in  many  cockpits.  Voice  communications  are  prevalent  both 
between  the  air  and  ground,  and  in  military  environments,  between  aircraft.  Checklists 
use  printed  media.  Communication  between  the  cockpit  and  the  flight  attendants  is  by 
telephone  on  most  commercial  airlines.  Clearly,  the  modem  commercial  and  military 
aircraft  utilize  multimedia  to  present  and  exchange  information  . 

The  situation  in  Air  Traffic  Control  is  similar.  Information  on  the  location  and 
identification  of  aircraft  in  the  airspace  is  shown  on  a  radar  screen.  Flight  strips  which 
contain  information  about  aircraft  are  still  present  in  printed  format,  Instmctions  to 
aircraft  and  interactions  with  aircraft  are  imparted  verbally.  Interaction  with  a  computer 
terminal  is  also  involved.  Again,  multimedia  are  involved. 

Despite  the  fact  that  the  use  of  multimedia  is  prevalent,  it  is  not  with  out  problems. 
Central  to  this  problem  is  the  fact  that  the  end  users  of  cockpits  —  the  pilots,  and  the  end 
users  of  ATC  systems  -  the  controllers,  must  interact.  However,  the  systems  which  they 
are  using  have  often  been  designed  in  isolation.  That  is,  designers  of  ATC  equipment 
and  designers  of  cockpits  are  most  often  two  independent  entities.  To  further  exacerbate 
the  situation,  even  within  the  cockpit  and  within  ATC  systems  design,  the  development 
and  testing  of  the  components  of  the  systems  have  often  been  independent.  The  use  of 
"off-the-shelf  components  is  hailed  by  many  to  be  a  great  time  and  money  saving 
endeavor.  However,  each  of  the  components,  operating  alone,  may  function  quite 
adequately,  incorporating  it  into  a  system  configuration  without  utilising  proper 
integration  studies  could  prove  disastrous. 


A  number  of  human  factors  aspects  need  to  be  addressed  in  detail  if  they  are  to  act  as 
effective  drivers  in  the  efforts  to  improve  aviation  safety.  In  addition  these  drivers 
should  be  used  in  a  proactive,  rather  than  reactive  way  and  the  multimedia  tools  used  to 
support  the  actions  must  be  appropriately  chosen  for  the  particular  activity. 

The  first  problem  which  arises  is  how  to  categorise  the  Human  Factors  actions  as  the  air 
and  ground  aspects  are  often  addressed  in  different  ways.  One  of  the  earliest  was  the 
definition  given  by  the  military’s  MANPRINT  (MANpower  and  PeRsonnel  INTegration) 
programme  This  essentially  takes  into  consideration  all  system  aspects  which  are 
concerned  with  Manpower,  Personnel,  Training,  Human  factors  engineering,  System 
safety.  Health  hazard  assessment. 

Organisational  and  management  aspects  can  be  added  as  many  aviation  organisations 
have  traditional  mechanistic  structures  which  need  to  be  modernised,  in  order  to  meet 
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modem  task  requirements  and  more  effectively  use  the  skills  and  expertise  of  the 
personnel. 

However,  more  appropriate  recent  conceptual  models  have  been  put  forward  in  order  to 
help  classification  of  work.  The  one  put  forward  by  ICAO  (Human  Factors  Digest  No.l) 
is  called  the  SHELL  model  from  the  terms  Software,  Hardware,  Environment  and 
Liveware.  This  notation  is  particularly  useful  in  identifying  the  usefulness  to  aviation  of 
HF  studies  as  HF  is  essentially  concerned  with  solving  practical  problems.  Thus  for 
Flight  Operations  the  specific  applications  are  categorised  as  but  not  limited  to: 

reduction  of  human  error 

training 

evaluation 

human  factors 

motivation 

flight  documentation 

flight  deck  design 

cabin  design 

visual  performance 

collision  avoidance. 

Eurocontrol  (1996)  have  built  on  the  SHELL  concept  to  specifically  address  ATC 
matters  and  consequently  categorise  specific  applications  as: 

Communication 
teamwork 
customer  focus 

rostering  and  working  practices 
human  factors  techniques 
career  and  personnel  development. 

Although  further  types  could  be  developed  it  is  important  at  this  stage,  to  concentrate  on 
the  essential  HF  disciplines  which  impact  on  aviation  safety.  Thus  the  ICAO  and 
Eurocontrol  categories  can  be  combined  as  follows: 
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HF  Category 

Meets  ICAO 
characteristics 

Meets  Eurocontrol 
characteristics 

Physical 

requirements 

Physical  size 
and  shape 

Physical 

needs 

Working 

practices 

Output 

characteristics 

Rostering  and 
working  practices 

Information 

processing 

Input 

characteristics 

Information 

processing 

Communication/Te 

amwork/Customer 

focus 

Organisational, 
regulation, 
management 
and  training 
aspects 

Environmental 

tolerances. 

Human  factors 

techniques 

(management) 

Career  and 

personnel 

development 

Working  practices 

Despite  the  fact  that  multimedia  have  been  extensively  involved  both  in  cockpits  and 
ATC,  the  problem  of  integration  of  all  the  information  and  required  functions  is 
pervasive.  Even  within  a  single  display  the  problem  of  integrating  all  of  the  information 
and  presentation  issues  can  be  problematic.  To  illustrate  the  problem,  in  a  recent  study 
by  Shaffer  and  Mills  (1993)  to  develop  guidelines  for  a  display  avionics  management 
unit  for  a  military  cargo  aircraft,  it  was  found  that  there  had  been  no  non-proprietary 
studies  that  addressed  the  interactions  of  menu  development  with  its  presentation, 
including  font  size,  contrast,  lighting,  location  and  display  density.  There  were 
numerous  studies  which  addressed  each  of  these  elements  in  isolation,  but  none  that 
addressed  the  issues  as  they  interact.  The  require  font  size  is  dependent  upon  the 
location  (in  this  case  in  the  cockpit)  which  determines  the  lighting  and  contrast 
requirements.  These  also  limit  the  amount  of  information  that  can  be  portrayed  on  the 
display  given  the  display  density  guidelines.  When  one  tries  to  make  trade-offs,  it  is 
difficult,  if  not  impossible  to  consider  all  of  the  interacting  factors  based  upon  today’s 
published  literature. 

Subsequent  to  this  effort,  Toms,  et  al  (1995)  performed  studies  at  Wright  Laboratories  in 
Ohio  using  simulated  flat  panel  displays  incorporating  the  best  “guestimate”  of 
combinations  of  display  characteristics.  The  question  being  asked  was  “is  the 
performance  with  the  new  display  as  good  or  better  than  performance  with  the  existing 
display?”  Since  existing  displays  were  being  utilised  by  crews  in  the  cockpit,  this  was 
considered  to  “validate”  their  usability.  However,  nothing  could  be  said  about 
optimising  performance,  or  whether  performance  might  have  been  even  better  using  a 
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slightly  different  configuration.  While  its  scope  was  limited  to  the  study  of  only  one  new 
configuration,  the  study  was  important  as  it  is  one  of  the  few  recent  studies  of  display 
integration  that  is  finding  its  way  into  the  published  literature. 

Most  studies  that  address  displays  as  an  integrated  part  of  a  total  system,  if  they  exist  at 
all,  are  considered  to  be  the  exclusive  property  of  the  researching  company.  As  a  result, 
each  researcher  must  “reinvent  the  wheel,”  and  the  body  of  literature  addressing 
integration  studies  remains  woefully  small.  Furthermore,  the  conduct  of  parametric 
studies  of  the  myriad  of  display  and  system  characteristics  which  interact  to  create  the 
end  product  is,  at  best,  an  Herculean  task.  This  task  probably  is  too  large,  and  cannot  be 
entirely  accomplished.  Nevertheless,  the  industry  and  the  traveling  public  would  be 
better  served  if  there  were  greater  sharing  of  information  among  the  researchers, 
particularly  where  integrated  systems  are  being  evaluated.  Or  perhaps  a  centre  for 
technology  exchange  should  be  instituted. 

New  solutions  for  existing  and  new  problems 

Models  and  case  studies  have  always  suffered  from  looking  backward  and  thus  solving 
yesterday’s  problems.  The  advent  of  modem  multimedia  techniques  suggests  that  a 
paradigm  shift  in  capabilities  has  now  occurred.  This  allows  us  to  realistically  model 
complex  l^ge  scale  system  whilst  allowing  individuals  to  specifically  to  address  their 
particular  area  at  whatever  level  of  complexity  is  appropriate  to  them. 

Obtaining  relevant  data 

Utilising  models  of  complex  systems  to  address  complex  issues  depend  upon  accurate, 
relevant,  and  sufficient  data.  These  data  should  be  derived  from  an  operational  context 
for  them  to  be  maximally  useful.  Fortunately,  the  miniaturization  of  video  cameras 
offers  a  unique  opportunity  to  obtain  such  data.  Empirical  tools,  using  video  offer  more 
reliable  and  vaUd  data  since  they  do  not  rely  upon  self-report  by  operators  on  what  they 
do  and  how  they  do  it.  Sometimes  such  self-report  reflects  what  people  think  they  do,  or 
what  they  think  they  are  supposed  to  do,  not  what  they  actually  do.  Models  using  such 
data,  are  therefore  of  limited  utility. 

Subjective  assessments  translate  into  the  “eye  of  the  beholder”,,  and  there  are  times  when 
such  a  view  is  not  complete.  A  case  in  point  is  revealed  in  a  study  on  the  stressful 
conditions  affecting  pilots  during  aircraft  carrier  landings  (Miller  et.  al„  1970).  The 
study  found  a  three  times  higher  level  of  serum  cortisol  levels  (indicating  stress)  even 
among  experienced  pilots.  This  occurred  despite  pilots  rating  themselves,  subjectively  as 
being  calm.  Ergo,  subjective  analysis  has  its  limits. 

An  Empirical  Methodology 

Video  using  Empirically  Validated  Task  Analysis  (EVTA)  offers  some  realistic  answers 
to  this  dilemma.  It  allows  a  presence  in  environments  where  observation  has  been 
impossible;  it  allows  one  to  replay  the  segments  to  determine  what  is  “really  going  on”; 
and  it  allows  one  to  verify  the  tasks  performed,  to  see  how  problems  are  faced  and  how 
they  are  successfully  and  sometimes  unsuccessfully  solved.  Because  the  camera  is 
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unobtrusive  (some  cameras  are  smaller  than  a  man’s  thumb),  the  operators  quickly  forget 
that  they  are  being  taped.  This  means  that  they  are  more  likely  to  behave  as  they 
normally  would,  not  as  they  think  they  should. 

The  camera  captures  images  and  conversation  on  videotape  that  may  be  viewed 
repeatedly  and  concurrently  with  the  time  consumed.  Provide  this  information  to  a  team 
of  qualified  observers  who  record  tasks  and  time  through  consensus,  and  there  is  a 
considerable  improvement  in  the  quality  of  empirical  data  derived. 

To  assist  in  the  analysis  process,  Paradigm  developed  the  proprietary  EVTA  software 
program.  The  EVTA  process  generates  a  task-descriptive  data  base  from  a  detailed 
analysis  of  operator’s  observable  activities  and  audible  communications.  In  some 
environments,  it  is  possible  to  also  record  the  information  that  is  presented  on  a  computer 
screen,  so  it  is  possible  to  determine  the  information  that  is  present  to  which  the  operator 
is  responding.  The  essential  features  of  the  EVTA  process  are  the  use  of  video/audio 
equipment  to  gather  permanent  records  of  operator  functions  from  operational 
environments,  and  the  generation  of  an  empirical  record  of  function  times  through  a 
software  package  which  builds,  manages  and  analyses  the  resulting  data  base.  The 
output  from  the  EVTA  software  is  intended  to  provide  the  data  required  by  and 
compatible  with  simulation  environments  such  as  Micro  SAINT  (Anon.,  1987,  SWAS 
(Holley  and  Parkes,  1987,)  and  HOS  (Lane  et  al,  1981).  More  recent  models  may  also 
use  such  data. 

The  focus  of  EVTA  is  on  observable  behaviours  such  as  hand  movements,  body 
movements,  internal  and  external  communications  and  visible  information  presented  on 
CRTs,  LCDs  or  other  idsplays.  Cognitive  activity  is  caputred  only  when 
communications  between  operators  or  an  overt  activity  indicate  that  it  has  occurred,  it  is 
not  inferred. 

Enhancements  to  the  process 

One  of  the  difficulties  with  using  data  obtained  video  analysis,  rests  with  the  analysis 
process  itself:  it  is  time  consuming;  tedious,  and  both  boring  and  demanding.  To 
address  some  of  these  problems,  (primarily  with  the  input  process),  a  new  system  was 
designed  by  Monterey  Technologies  Inc.  for  the  U.S.  Navy:  the  Video  Information 
Extraction  Work  Station  (VIEWS).  This  system  digitizes  the  audio/video  tape  which 
allows  for  random  access  anywhere  on  the  audio/video  stream  without  the  time 
consuming,  inaccuracies  of  rewinding  of  tapes.  Start  and  stop  times  can  be  entered  with 
a  single  keystroke,  eliminating  the  necessity  for  entering  the  individual  numbers  in  the 
time  hack  thereby  reducing  errors.  Multiple  sources  of  data  can  be  entered.  Digital 
information  such  as  eye  movments  can  be  integrated  with  information  from  the 
audio/video  stream  on  the  same  timeline  It  is  possible  to  zoom  into  a  portion  of  a  video 
frame  for  closer  examination.  The  audio/video  stream  can  be  viewed  on  a  frame  by 
frame  basis,  if  very  precise  time  information  is  required.  One  of  the  most  useful  features 
is  the  loop  capability,  which  allows  the  analyst  to  identify  the  start  and  stop  of  an  event 
and  to  replay  the  loop  repeatedly  until  all  of  the  information  about  the  event  is  recorded. 
This  is  particularly  useful  in  deciphering  garbled  communications  which  frequently 
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occur  in  operational  environments  particularly  in  aircraft.  Finally,  future  versions  of  the 
VIEWS  system  will  include  the  ability  to  enhance  images  and  audio  quality. 

The  only  study  performed  to  date  using  data  obtained  from  VIEWS  is  a  study  for  a  major 
telecommunications  firm  which  addressed  a  detailed  job  analysis  of  customer  service 
representatives.  That  study  used  a  prototype  version  of  VIEWS  and  despite  its  pre- 
production  bugs,  it  offered  some  assistance  in  translating  multi-media  source  information 
into  useable  data.  As  the  technology  matures,  these  types  of  data  can  be  useful  in 
modeling  complex  systems  that  are  becoming  commonplace. 

Information  processing 

In  many  cases  it  is  not  possible  to  engage  in  in-depth  studies  such  as  those  involving 
EVTA  and  VIEWS  and  data  must  be  provided  in  the  most  effective  manner  that  expert 
opinion  feels  is  appropriate.  One  such  area  is  information  processing.  This  is,  of  course 
a  major  activity  at  present,  due  to  the  immense  amount  of  information  which  is  available 
and  relevant  to  every  job.  As  a  safety  critical  activity,  the  aviation  field  has  always  been 
in  need  of  current  information  to  allow  for  timely  processing,  as  so  many  aspects  of  the 
flight  environment  change  rapidly,  and  flying  is  a  notoriously  unforgiving  environment 
for  errors.  Fortunately,  multimedia  techniques  now  give  us  an  opportunity  to  provide 
instant  communications  of  whatever  information  is  available  and  appropriate  to  the 
situation.  Because  of  advanced  technology,  it  is  possible  to  present  information  in  a 
hierarchical  fashion:  the  most  important  information  provided  first  and  foremost.  This 
information  can  be  presented  in  a  myriad  of  ways.  The  most  appropriate  media  can  be 
selected  for  the  particular  application,  with  the  ultimate  goal  of  reducing  errors. 

Application  to  training 

Applications  to  information  processing  are  well  understood  in  the  operational  context, 
however,  the  real  challenge  is  to  transfer  the  capability  into  the  off-line  activities  such  as 
training,  where  traditionally  the  demands  have  often  not  been  so  stringent.  There  are 
several  aspects  of  these  off-line  activities  that  deserve  special  attention:  ensuring  up-to- 
date  information  in  the  airspace  environment  and  maintanence  of  up-to-date  instructors., 
training  time,  and  controller  education  for  new  technologies. 

1.  Recieving  appropriate  information  on  characteristics  of  new  aircraft  entering  an 
airspace. 

Keeping  up-to-date  on  all  of  the  relevant  information  about  new  aircraft  is  traditionally  a 
problem  in  developing  countries,  and  can,  because  of  volume,  become  a  burden  even  to 
large  counries.  The  ISO  9000  series  stresses  that  as  a  part  of  the  quality  management 
process,  it  is  critical  that  relevant  information  must  be  provided  to  the  actual  user  and  not 
left  in  the  manager’s  office. 

With  current  multimedia  techniques  this  information  can  be  placed  directly  with  the 
actual  user  by  using  CD-ROM  or  via  the  Internet  thus  providing  on  the  job  training. 
Adding  to  this,  the  realistic  use  of  video,  there  can  be  a  reinforced  connection  between 
the  type  of  aircraft  and  its  performance.  This  can  be  provided  at  the  level  of  detail 
required  by  the  specific  user. 
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2.  Maintaining  up-to-date  instructors. 

When  the  instructional  staff  must  be  removed  from  their  operational  capacities,  it  is  is 
difficult,  particularly  over  time,  for  them  to  maintain  their  currencies.  Special  skills  can 
be  lost,  and  up-to-date  examples  can  be  sacrificed.  This  problem  applies  particularly  to 
controllers  but  also  to  airhne  pilots.  While  simulations  are  useful  in  providing  hands-on 
experience  particularly  with  pilot  training,  the  importance  of  having  instructors  with  up- 
to-date,  current ,  and  extensive  expertise  is  invaluable.  With  the  availability  of  Web  sites 
on  the  Internet,  such  information  can  be  documented  and  available  to  trainees  from 
diverse  geographic  localities. 

3.  Training  time 

Training  time,  particularly  for  controllers  has  always  been  considered  a  lengthy  process, 
and  many  countries  are  making  great  efforts  to  reduce  the  time.  New  techniques  include 
the  use  of  PCs  to  match  training  presentation  to  individual  student  learning  styles. 
Simulator  exercises  tailored  both  to  the  geographic  area  (with  its  accompanying  level  of 
complexity)  and  to  the  student  learning  styles  provide  a  cost  effective  and  timely  training 
philosophy. 

4.  Contoller  education  for  new  technologies 

With  the  introduction  of  the  Advance  Automation  Techniques  (AAT)  into  ATC  there  are 
questions  regarding  the  extent  to  which  the  present  generation  of  controllers  are 
adequately  educated  to  understand  the  understanding  principals  of  the  new  technology 
and  the  imphcations  of  its  practical  implementation.  Both  CD-ROM  and  the  Internet 
offer  appropriate  and  cost-effective  training  in  these  new  techniques.  Self-learmng  can 
be  a  powerful  tool  for  providing  currencies,  however  it  should  be  noted,  that,  given  a 
variety  of  learning  styles  and  management  incentives,  not  all  of  the  current  controllers 
will  respond  appropriately  to  such  techniques.  A  vehicle  to  identify  those  controllers 
who  might  “fall  between  Ae  cracks”  and  not  respond  appropriately  should  be  developed. 
For  those  controllers,  more  conventional  techniques  of  classroom  training  may  be 
necessary. 

Other  applications 

There  are  many  other  applications  which  are  reciving  attention  and  the  present  situation 
will  make  for  an  excellent  discussion.  Among  some  of  the  topics  must  be  mentioned: 

1.  The  ability  to  strengthen  the  data,  information  and  knowledge  links  between 
regulatory  authorities,  licencing  authorities  and  operational  organisations  (eg.  for 
monitoring,  control  and  recording  of  results). 

2.  The  abihty  to  link  simulators  to  CBT  and  thus  improve  trainnig  programmes  for 
standard  training  or  for  re- validating  licenses). 

3.  The  ability  to  link  flying  simulators  and  ATC  simulators  and  thus  practice 
realistic  cockpit  skills.  Pilots,  ATC  skills  and  to  produce  CRM  scenarios. 
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Management  and  future  challenges. 

Management  must  always  be  concerned  with  the  competency  and  currency  of  its  staff. 
Given  the  competitive  marketplace  present  in  the  aviation  community,  this  presents 
constant  challenges.  Adequate  and  appropriate  communications  are  essential  for  all. 
Simulations  (using  PCs  and  more  sophisticated  computers),  CD  ROM,  the  Internet, 
Computer  Based  Training  (CBT)  all  offer  appropriate  technological  solutions  to  modem 
aviation  problems. 

However,  the  challenge  is  for  the  multimedia  models  to  stay  current  with  the  real  systems 
and  the  real  problems  and  thus  to  be  continuously  looking  forward.  This  is  the  challenge 
for  management.  This  then  is  central  to  establishing  and  ensuring  the  benefits  which 
multimedia  techniques  can  bring  to  the  aviation  system  in  order  to  create  an  effective 
and  systematic  integrated  approach  to  the  human  performance  interactions  in  the  cockpit 
and  air  traffic  control.  This  is  a  holistic  approach  that  affords  the  opportunity  for 
everyone  to  stay  abreast  of  the  latest  technology. 

This  is  both  the  opportunity  and  the  challenge  for  integrating  technology  and  multimedia. 
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Introduction 

The  plannings  of  the  ICAO  (International  Civil  Aviation  Organsiation)  concerning  the 
global  implementation  of  future  communication,  navigation  and  surveillance,  and  air 
traffic  management  require  worldwide  operational  satellite  based  systems  (Global 
Navigation  Satellite  Systems,  GNSS)  [ICAO,  1988].  By  the  use  of  satellites  for 
navigation  and  data  communication  purposes  it  will  be  possible  to  create  surveillance 
(Automatic  Dependent  Surveillance,  ADS)  for  air  traffic  control  (ATC)  independent  of 
ground  derived  radar  equipments.  Based  on  a  two-way  air-ground  data  transmission  a 
variety  of  onboard  generated  data,  e.g.  4D  data  of  flight  profile  or  current  weather  data 
can  be  included  in  the  planning  and  supervision  process  of  air  traffic  control  refering  to 
its  optimization.  The  improvement  of  quantity  and  quality  of  data  however  requires  a 
redesign  of  the  previous  human-machine  system  ^controller  work  position^  (CWP)  and  a 
modification  of  the  ATC  procedures  [ICAO  1995,  EUROCONTROL  1991  und  1994]. 

In  the  context  of  a  project  promoted  by  Deutsche  Forschungsgemeinschaft  research  was 
carried  out  to  this  at  the  Institute  of  Aeronautics  and  Astronautics  at  the  Berlin 
University  of  Technology.  The  examinations  reflect  to  operative  and  cognitive  aspects  of 
the  air  traffic  controller's  work  using  the  application  of  ADS.  The  aim  was  to  get  a  user 
oriented  design  of  the  human  machine  system,  and  thus  to  specify  necessary 
modifications  of  modes  of  operation  used  at  present  by  air  traffic  control. 

In  a  first  approach  the  preliminary  examinations  were  concentrated  on  the  analysis  of 
present  air  traffic  control  systems  as  well  as  on  the  task  analysis  of  the  controller's  work. 
A  complex  task  scheme  of  the  radar  controller,  the  coordinator  and  their  team  work  was 
developed  based  on  the  Goals  Operator's  Methods  Selection  Rules  -  model  [Card  et.al., 
1983]  declared  itself  through  software  ergonomy.  Furthermore  a  knowledge  acquisition 
was  carried  out  for  the  conflict  management  process  to  get  information  about  the 
subjective  theories  of  the  controllers  regarding  their  main  task  of  conflict  detectionand 
solution  as  well  as  their  mental  processing.  The  knowledge  about  cognitive  processes  of 
the  controllers  completed  the  results  of  the  task  analysis.  The  knowledge  acquisition 
proved  as  meaningful  and  necessary  with  regard  to  the  implementation  of  numerous  new 
information  particularly  into  the  ADS  CWP.  This  scheme  (task  analysis  and  knowledge 
acquisition)  formed  a  basis  for  the  design  of  the  future  ADS  CWP.  It  was  valid  to  answer 
the  question  how  the  technically  possible  availability  of  new  data  has  been  brought  into 
harmony  to  the  required  data  for  previous  work  tasks  as  well  as  the  construction  of 
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subjective  theories  of  the  controllers. 

The  development  of  requirements  for  the  specification  of  the  human-machine  interface  of 
the  CWP  then  was  carried  out  in  a  second  approach  as  well  as  the  development  of  the 
graphical  dialog  interfaces  and  the  implementation  in  a  simulation  environment.  At  least 
the  complex  human-machine  interface  of  the  ADS  CWP  was  evaluted  heuristically  by 
experts  of  interface  design  and  air  traffic  controllers.  So,  an  evaluated  ADS  CWP  for 
further  experimental,  comparative  examinations  of  possible  ADS  modes  of  operation 
exists. 

Preliminary  Examination 

The  analysis  of  the  present  situation  covered  documents  such  as  ODID  IV  (Operator 
Display  and  Input  Development)  [EUROCONTROL,  1994]  and  the  design  guidelines  for 
controller  work  position  appropriate  to  COPS  (Common  Operational  Performance 
Specification)  [EUROCONTROL,  1991].  Observations  and  interviews  of  controllers  at 
the  ACC  Tempelhof  of  the  Deutsche  Flugsicherung  GmbH  (DFS)  were  also  carried  out. 
The  aim  has  been  the  analysis  of  different  design  concepts  as  well  as  the  inquiry  of  the 
current  state  of  development  of  air  traffic  controller  work  positions  like  DERD-XL 
(representation  of  extracted  radar  data/extended  version).  Potential  design  lacks  and 
advantages  of  present  air  traffic  control  systems  should  be  recognized. 

20  experienced  radar  controllers  (EC)  and  coordinators  (CC)  took  part  in  the  interview. 
They  could  refer  to  an  average  professional  experience  of  nine  years  and  had  admittances 
for  at  least  two  and  maximum  twelve  air  traffic  control  sectors.  Ratings  and  open 
questions  formed  the  main  emphasis  of  the  questionare.  These  were  useful  to  judge  the 
design  of  the  single  control  units  and  informational  displays  as  well  as  the  expected  benefit 
of  possible  innovations. 

The  controllers  named  a  variety  of  CWP  shortages  and  estimated  the  general  use  of  data 
link  for  the  air-ground  communication  as  very  desirable.  An  expansion  of  the  present 
information  according  to  the  aircrafts  to  be  supervised  seemed  to  be  considered  very 
positive.  An  absolutely  required  attention  of  design  principles  became  in  connection  with 
this  emphasis,  too.  Almost  all  of  the  controllers  reported  that  they  did  not  have  sufficent 
support  for  the  conflict  detection  and  solution  process  by  the  present  operational  system. 
Need  for  action  would  particularly  exists  in  the  prediction  of  potential  conflicts  between 
aircrafts.  One  solution  to  a  variety  of  the  problems  wOl  be  possible  by  the  use  of  data  as 
specified  by  ADS.  By  the  use  of  data  link  safety  in  air  transportation  will  increase  by  a 
more  precise  identification  of  flight  targets,  one  improved  supervision  of  airplanes  in  areas 
with  no  or  less  enlarged  radar  equipment  as  well  as  one  enlarged  quantity  of  information. 
A  better  u.sage  of  airspace  capacity  will  be  possible  by  reducing  the  separation  standards. 
In  addition,  the  correlation  of  the  scheduled  flight  data  with  onboard  generated  position 
and  flight  intention  data  will  be  a  relief  for  the  execution  of  planning  and  coordination 
tasks. 

Further  examination  was  carried  out  for  completion  of  the  analysis  of  present  operative  air 
traffic  control  systems  regarding  the  current  and  future  state  of  development  to 
functionality  and  operability  of  ADS  in  air  traffic  control.  This  served  to  the  inquiry  of 
perhaps  available  specifications  for  the  ADS  functionality  respectively  its  application  into 
future  air  traffic  control  system.  No  standardization  for  ADS  was  carried  out  by 
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the  ICAO  [ICAO,  1995]  until  today.  The  exchange  of  ADS  data  between  air  traffic 
control  and  aircrafts  will  be  initiated  by  any  of  them.  This  can  be  earned  out  periodically 
or  on  request.  This  so  called  ADS  report  can  contain  up  to  eight  blocks  of  data  of  various 
information.  The  composition  varies  in  dependence  of  the  avionic  standard,  the  volume 
of  air  traffic  and  operational  aspects.  The  transferred  data  refer  e.g  to  the  position  of  the 
aircraft  (basic  block),  the  current  speed  and  heading  (ground/air  vector),  the  4D-profile 
calculated  by  a  flight  management  system  (projected  profiles)  or  actual  flight  weather 
data  (met  info). 

A  detailed  task  analysis  of  controller's  work  in  the  context  of  the  first  work  package 
became  necessary  because  it  was  not  available  at  the  beginning  of  the  project.  However, 
the  description  of  possible  modifications  of  future  ATC  procedures  given  by  the 
introduction  of  ADS  and  the  redesign  of  an  ADS  CWP  required  some  task  analysis 
(Hollnagel,  1990). 

For  this  reason  a  document  analysis  has  been  conducted  based  on  the  BAFVK 
(operational  procedures  for  air  traffic  service)  and  the  EUROCONTROL  Task  Book 
AF51  (EUROCONTROL,  1995)  as  well  as  further  observations  and  interviews  with 
selected  ATC  experts  at  the  ACC  Tempelhof  of  the  Deutsche  Flugsicherung  GmbH.  The 
result  was  a  clear  structure  of  tasks  to  the  present  work  of  radar  controllers  and 
coordinators  in  connection  with  the  used  DERD-XL  work  station.  To  be  able  to  better 
describe  the  distribution  of  tasks  in  the  human-machine  system,  a  customization  of  the 
determined  tasks  to  the  sooner  mechanistic  GOMS  model,  known  from  the  software 
development  discipline,  was  carried  out  on  a  functional  level. 

As  a  matter  of  priority  observable  shares  to  the  very  complex  task  of  air  traffic 
controller's  supervision  were  made  visible  with  the  task  analysis  already  mentioned.  It 
was  valid  to  got  a  picture  of  this  work  as  complete  as  possible  including  also  the  rules  to 
the  practice  of  the  determined  tasks  being  based  and  therefore  subjective  theories  of  the 
controllers.  In  an  extensive,  methodically  complex  examination  with  20  subjects 
(controllers  of  ACC  Tempelhof)  the  main  task  of  the  controllers,  the  conflict 
management,  was  analyzed.  The  methods  used  were  an  especially  designed  questionare, 
the  method  of  network  development  and  the  method  of  constructive  interaction. 

The  subjects  spread  out  as  follows  on  the  single  methods: 


controller 

/method 

network 

questionare 

constructive  interaction 

network 
new  info 

network 
known  info 

questionare 

detection 

questionare 

resolution 

type  of 
conflict  1-4 

type  of 
conflict  5-9 

EC 

7 

10 

5 

6 

5 

5 

CC 

10 

8 

5 

6 

5 

5 

Table  2.1:  Distribution  of  subjects 

The  questionare  used  in  the  test  consisted  of  so-called  open  questions  and  of  ratings  with 
main  emphases  concerning  the  demographic  data,  the  conflict  detection,  and  resolution. 

Using  the  method  of  network  development  the  test  participants  had  the  task  to  group  a 
set  of  data  cards  containing  information  will  need  at  present  and  future  for  the  practice  of 
their  job  as  air  traffic  controller.  In  a  first  step  they  had  to  sort  all  cards  so  that  all 
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information  which  has  a  similar  importance  for  the  conflict  detection  process  were 
summarized  in  a  group.  Then  they  should  try  to  structure  these  cards  in  a  network,  which 
represents  individual  mental  information  structure  of  their  personal  conflict  detection 
process. 

Due  to  the  number  of  subjects  in  comparison  with  the  examined  parameters  (number  of 
information  to  sort)  an  exclusively  qualitative  analysis  of  the  obtained  data  in  accordance 
with  the  network  development  seemed  to  be  efficient  [Liier,  1987].  A  quantitative 
comparison  did  not  seem  meaningful  here.  Observations  during  the  execution  of  the 
method  showed  that  controllers  had  some  difficulties  to  structure  the  information  units  of 
such  a  highly  automated  task  like  conflict  detection.  The  controllers  consequently 
emphasized  the  missing  general  validity  of  their  networks.  This  was  caused  by  the  variety 
of  conditions  and  situation  variables  having  an  effect  on  the  process  of  conflict  detection 
which  were  not  checked  explicitly.  Nevertheless  relatively  similar  results  were  produced 
within  the  respective  random  sample  and  the  low  number  of  subjects.  This  points  out  that 
there  are  generally  pure  coherences  within  the  conflict  detection  process. 

Controllers  were  confronted  with  the  method  of  constructive  interaction  (Nielsen,  1993)  in 
the  context  of  the  examinations  at  the  ACC  Tempelhof.  The  potential  conflict  scenarios 
consisted  of  static  variants  designed  as  abstract  radar  pictures  and  control  stripes.  The 
conflict  detectionand  resolution  was  carried  out  during  the  series  of  experiments  in  a  team 
of  radar  controllers  and  coordinators  according  to  the  organization  in  ATC.  The  task  of 
the  controller  was  to  detect  and  solute  potential  conflict  situations  based  on  the 
information  sources  already  mentioned  (radar  picture  and  control  stripes).  For  the  series 
of  tests  nine  defined  conflict  types  have  been  taken  into  account  according  to 
EUROCONTROL  Task  Book  (EUROCONTROL,  1995).  The  conflict  scenarios  were 
formed  on  the  radar  sector  UR  2  in  the  upper  airspace  of  the  UIR  Berlin  and  checked  in 
cooperation  with  active  controllers  with  regard  to  its  ecological  validity.  It  had  to  be 
stated,  that  coordinators  and  radar  controllers  primary  used  flight  level  changes 
independently  of  the  conflict  type  and  secondarily  measures  affected  the  lateral  profile  like 
"parallel!  offset"  and  "directs".  The  decision  about  the  application  of  one  of  the  actions 
mentioned  was  dependent  on  the  flight  performance,  particularly  the  speed  of  the  aircrafts 
concerned,  and  the  accordance  of  the  partial  routings  of  the  conflict  partners  concerned. 

By  using  an  extensive  catalog  of  methods  a  complex  picture  of  the  mental  processes  of 
conflict  detection  and  resolution  as  well  as  a  variety  of  specific  controller "s  knowledge  had 
been  determined.  Because  of  the  varying  efficiency  of  the  employed  methods  the  single 
results  were  summarized  with  the  intention  to  get  extensive  knowledge  of  information 
contents  and  their  structuring  at  the  human-machine  interfaces  of  the  CWP. 

The  resulting  specification  of  requirements  (see  chapter  3.1)  became  the  guideline  for  the 
further  development  of  the  human-machine  interfaces  of  the  ADS  CWP.  Main  emphases 
were  made  concerning  the  primary  tasks  of  controllers  e.g.  conflict  detection  and 
resolution  and  the  design  of  tools  for  controller  assistance. 

Design  of  the  HMI  of  a  controller  working  position  with  ADS-functionality 
Specification  of  the  HMI 

Resulting  from  the  status  analysis  and  the  pre-studies,  a  catalog  of  requirements  for 


106 


designing  the  HMI  of  the  ADS  CWP  was  defined.  The  catalog  is  influenced  by  the 
guidelines  of  ICAO  (1995),  international  guides  for  display  design  especially  the  ISO 
9241  part  10  and  part  12,  the  DIN  66234,  part  8,  the  results  of  the  ODID-Group  and 
COPS-Specification  from  1991,  and  topical  research  results  of  design  of  GUIs  like  the 
study  of  Charwat  (1996). 

Besides  this  analysis  of  documents,  the  results  of  described  status  analysis  and  the  pre¬ 
study  already  mentioned  (task  analysis  and  knowledge  acquisition)  were  used  to  develop 
the  catalog.  The  results  of  knowledge  acquisition  and  especially  the  conflict  management 
in  a  team  of  controllers  had  have  a  great  influence  on  the  design  process.  Due  to  the 
collected  results  it  was  possible  to  fix  a  lot  of  important  information  have  to  be  shown  on 
the  ADS  CWP.  Furthermore  a  number  of  conflict  resolutions  prefered  by  controllers 
could  be  recorded. 

The  requirement  catalogue  defines  following  rules  of  designing  the  ADS  CWP: 

1.  Realization  of  technical  requirements  of  air  traffic  control  (ATC): 

•  Realdisplaying  of  airspace  (map  data,  airspace  structure), 

.  Real  fli^tplan  data  of  aircrafts  with  standard  avionik  and  ADS  specific  avionics 
(plan  data  and/or  radar  data  are  at  controllers  disposal)  oriented  by  the  ICAO 
standard  of  transmission  rates  of  ADS  reports  in  high  density  areas  (1995), 

•  Exact  conflict  detections, 

•  ADS  specific  service  functions  e.g.  displaying  of  planned  flight  profile,  ADS 
events,  and  meteorological  data, 

•  Calculation  and  development  of  complete  conflict  resolutions  which  are  possible 
by  automated  techniques  and  be  prefered  by  controllers. 

2,  Support  of  typical  job  tasks  of  air  traffic  controllers  as  well  as  their  ATC 

functions  defined  by  COPS 

2.1. General  requirements 

•  Unique  display  for  radar  controllers  and  coordinators. 

•  Possibility  to  adjust  the  interface  to  personal  preferences. 

•  Adjustments  have  to  be  saved  by  the  system  under  a  special  personal  code. 

•  Possibility  to  execute  all  typically  defined  job  tasks. 

The  ADS  CWP  has  to  hold  the  following  information  and  functionalities  (part  2.2.  -  2.6.) 
to  the  controllers  disposal  in  view  of  the  functions  of  controllers  defined  by  COPS. 

2.2.  Planning  functions 

•  Presentation  of  the  complete  flight  information  by  electronical  stripes  (sorted  by 
time  and  deleted  automatically;  the  stripes  have  to  show  at  least  the  following 
informations:  AC  code,  AC  type,  sector  entry  point,  entry  time,  and  entry  level, 
sector  exit  point,  exit  level,  and  exit  time  as  well  as  the  planned  flight  profile  with 
at  most  five  waypoints  inside  of  the  bordering  sector.  Graphical  flight  path 
elements  (lateral  and  vertical)  should  be  shown  on  the  central  radar  screen,  likewise 
outside  the  controlled  sector,). 

•  Facility  of  a  simulation  of  flight  path  changes  with  a  following-up  check  for 
conflict  freedom, 

•  A  long  term  conflict  alert  with  an  exact  visualization  of  conflict  partners  and 
reasons. 
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•  Holding  possible  conflict  resolutions  to  controllers  disposal,  and  displaying  them  in 
an  understandable  way. 

2.3.  Surveillance  functions 

•  Presentation  of  the  traffic  situation  of  the  controlled  sector  as  well  as  the  adjacent 
sectors  (with  a  target  symbol  and  the  accompanying  label,  and  at  least  the  following 
informations:  call  sign,  topical  flight  level,  figure  of  merit,  AC  type,  AC  category). 

•  Conflict  warning  with  a  preview  of  more  than  two  minutes  (containing  the  reason  of 
conflict  and  the  conflict  partners). 

.  Presentation  of  the  following  planned  flight  profile  in  a  schedule  and  by  means  of 
graphical  elements  with  the  following  information:  sector  exit  point,  exit  time,  and 
exit  level). 

•  Supporting  the  orientation  of  controllers  in  the  airspace  by  a  flexible  measurement 
vector  and  an  understandable  presentation  of  the  vertical  profile. 

2.4.  Monitoring  and  control  functions 

.  A  workload  display. 

•  Status  display  of  the  particular  modules  e.g.  conflict  control. 

•  Implementation  of  a  zoom  function  and  the  possibility  to  adjust  the  picture. 

2.5.  Coordination  within  the  controlled  sector,  within  the  ACC,  and  between 
other  ACCs  (was  not  the  topic  of  the  project  at  present). 

2.6.  Air-ground  Communication  (was  not  the  topic  of  the  project  at  present), 

2.7.  With  the  help  of  ADS  CWP  the  controllers  should  be  able  to  detect  and 
process  the  following  conflict  scenarios: 

.  high  conflict  potential  caused  by  the  structure  of  sector, 

•  potential  conflicts  resulting  from  aircraft  movements, 

•  conflicts  caused  by  weather  conditions, 

•  emergencies. 

2.8.  Observance  of  general  software-ergonomic  and  cognitive-ergonomic 
principles 

2.9.  System  architecture 

•  The  simulation  should  have  been  implemented  in  a  UNIX  environment. 

•  It  should  be  designed  according  to  the  chent  server  model  to  bear  sufficient 
flexibility. 

•  The  simulation  should  have  a  minimum  of  dead  times. 

•  The  controllers  should  be  able  to  control  a  lot  of  traffic  (approximately  60  AC  s 
per  hour)  with  ADS  CWP. 

•  Operations  using  a  mouse  as  input  device. 

.  Little  keyboard  usage  (e.g.  for  input  of  controller  codes). 

Implementation  into  a  simulation  environment 

The  software  part  of  the  ADS  CWP  discussed  here  is  a  development  from  an  experimental 
ATC  simulation  produced  by  the  DFG  project  “Fluglotsenarbeitsplatze”  (AZ.:  Fr375/45- 
l).  It  is  implemented  on  Unix  workstations  using  the  client  server  model  and  X- Windows. 
The  server  simulates  the  radar  turn,  keeps  all  flight  plans,  flight  profiles,  ADS  data,  and 
controls  inter  process  communication. 
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The  following  figure  3.1  shows  the  communication  scheme  for  the  ADS  CWP  and  other 
clients: 


Client: 
Air  traffic 
simulation 


> 


Server 


Client  ADS  CWP 


Client 

METEC 


Client 

COREC 


Figure  3,1:  Communication  scheme  for  ADS  CWP 

contains;  ADS:  ADS  database,  FPL:  flight  plan  database,  METEC:  conflict  detection, 
COREC:  conflict  solution 


To  improve  performance,  the  CWP  client  was  divided  into  three  subprocesses,  separating 
the  communication  with  the  server  from  its  radar  driver  (calculation  of  aircraft  positions 
and  zoom  functions)  and  frontend  (visualization  of  air  traffic  situation  display  and  user 
interaction). 

Besides,  in  the  ADS  CWP,  all  mouse  clicks  can  be  recorded  using  log  file  protocols.  This 
registration  method  allows  immediate  reproducability  of  all  actions  performed  by  the 
subjects,  and  thus  makes  it  possible  to  conduct  exact  fault  analysis,  define  reaction  times, 
and  to  reconstruct  all  activity  sequences.  Log  file  protocolling  is  a  substantial  addition  to 
interviews  and  video  recordings  as  frequently  used  evaluation  methods. 

Development  of  the  Graphical  User  Interface 

At  the  beginning  of  this  project  there  was  no  suitable  ADS  CWP  prototype  for  simulation 
purposes.  So  the  development  of  a  new  controller  work  position  became  a  main  part. 
This  CWP  then  could  be  used  as  a  test  bed  for  the  evaluation  of  the  Computer  Human 
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Interface  (CHI). 


The  foUowing  figure  depicts  the  structure  of  the  CHI  and  its  essential  features.  The 
frontend  was  built  following  the  rapid  prototyping  paradigm  with  Tcl/Tk.  This  interpreter 
language  comes  with  a  lot  of  widgets  which  are  good  suited  for  developing  Graphical 
User  Interfaces  under  the  X  windows  system. 


Pull  down  menu  contains:  CWP  setup, 
flight  plan  setup,  ADS  data  setup, 
conflict  module,  communication  fiel 
and  library 


priority  setup  is  used  for  preselection  of 
special  conflict  solutions  depending  on 
sector  load  by  the  controller 


vertical  window 
used  to  edit  the 
vertical  profile 


radar  screen  with:  sector  structure, 
restricted  areas,  air  ways,  way 
points,  air  craft  symbols,  extentable 
labels,  context  menu,  graphical 
editor,  etc. 


context  sensitive  multi  function  bar 
to  enter  the  graphical  editor, 
activating  the  vertical  window  and 
assigning  or  testing  a  modified 


Figure  3.2;  Frontend  of  the  ADS  CWP 


The  frontend  is  sub-structured  in  five  main  blocks:  radar  screen,  pull  down  menu,  context 
sensitive  bar,  info  Box,  and  work  load  display.  The  radar  screen  with  all  its  relevant 
information  especially  for  short  term  management  is  placed  in  the  center  of  the  user 
interface.  All  additional  information  for  instance  flight  plans  or  early  detected  conflicts  will 
be  accumulated  in  the  Info  Box.  The  work  load  widget  displays  the  expected  work  load 
within  next  twenty  minutes  which  is  calculated  by  considering  the  given  flight  plans  of 
the  scenario. 

Setup  functionality  is  provided  by  the  pulldown  menu.  The  status  bar  is  placed  at  the 
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bottom  of  the  screen.  It  permanently  shows  a  scale  of  15  NM  based  on  the  actual  zoom 
factor.  Additionally  it  contains  the  buttons  of  the  graphical  editor  during  edit  mode.  This 
editor  allows  editing  of  4D  flight  profiles. 

The  CWP  should  be  usable  for  aircrafts  supporting  different  avionic  standards.  This  is 
why  context  menus  and  labels  of  aircrafts  contain  different  information.  The  following 
table  summarizes  this  special  information  for  the  labels  of  ADS  compatible  and  standard 
air  crafts. 


ACtvoe 

aircraft  label 

Explanation  of  Abbreviations 

Standard 

ID,  AC  Type, 

ID 

AC  code 

Ti.  FI  act.  FI  koor,  NexWP,  FI  NexWP,  At,  TAS; 

AC  Type 

AC  type  identifier 

ti 

clirab/descent  indicator 

Fiact 

actual  height 

FI  coor 

coordinated  height 

NexWP 

next  (planned)  way  point 

ADS 

ID,ACType,FOM; 

FI  NexWP 

height  at  next  way  point 

Ti,  Rates,  FI  act,  FI 

coor,  NexWP,  FI  NexWP,  At,  TAS; 

At 

time  to  next  way  point 

wind  direction,  wind  speed,  turbulence,  DAT 

TAS 

True  Air  Speed 

. 

FOM 

data  quality  of  ADS  avionic 

3  line  is  activable 

Rates 

rate  of  climb/descent 

exclusivly  for  ADS 

OAT 

Outer  Air  Temperature 

aircrafts. 

Table  3.1:  Information  of  aircraft  labels 


The  ADS  CWP  frontend  was  extended  by  a  graphical  editor  which  was  created  in  this 
project.  It  is  partially  integrated  in  the  radar  screen  (lateral  profile).  The  part  of  editing 
vertical  profiles  is  implemented  in  a  separate  window  which  is  activated  on  demand  by 
the  controller. 

The  editor  allows  modification  of  profiles  without  intervention  of  keyboard  inputs.  The 
modified  and  acknowledged  profiles  will  be  passed  to  the  air  traffic  simulation  and  to  the 
conflict  detection  module.  This  higher  dynamic  behaviour  of  the  system  allows  a  real 
time  management  near  to  reality. 
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Figure  3.3:  Graphical  editor  for  modification  of  lateral  profiles 

The  conflict  management  system  is  an  additional  client  which  is  communicating  with  the 
ADS  CWP.  It  consists  of  a  conflict  detection  module  (METEC),  a  conflict  solution 
module  (COREC)  and  a  priority  controller  which  is  used  for  a  user  defined  preselection  of 
possible  solutions.  The  detection  module  is  configured  by  the  pulldown  menu  (left  part  of 
following  figure  3.4).  The  visualization  of  conflict  regions  and  additional  information  are 
on  the  radar  screen  too  (right  part).  The  module  for  conflict  solution  is  triggered 
automatically  for  the  earliest  conflict  or  manually  for  a  specified  one. 
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Figure  3.4:  Configuring  the  conflict  detection  (left)  and  visualization  (right)  of  a  conflict  situation 


Heuristic  Analysis  of  the  ADS  CWP  Prototype 
Description  of  the  used  method 

After  last  system  tests,  a  heuristic  analysis  of  the  interface  of  ADS  CWP  was  conducted 
in  May  1997.  The  subjects  were  experts  of  air  traffic  control  as  well  as  experts  for 
interface  design.  The  task  was  to  judge  the  whole  functionality  of  the  working  position, 
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the  monitoring  of  infomiation,  and  the  ability  to  manipulate  buttons  and  other  input 
devices.  The  qualitative  evaluation  was  oriented  by  heuristics  which  are  well  known  in 
literature  (compare  with  Nielsen  and  Molich,  1990).  So  it  was  interesting  to  analyse  the 
dialog,  the  reduction  of  memory  strain,  and  possible  faults  of  graphical  presentation  like 
wrong  font  size,  errors  in  layout  or  trouble  with  controling  the  interface. 

Before  the  test  begun,  the  subjects  had  the  possibility  to  learn  about  the  interface  and  its 
functions  and  could  „play“  with  it.  Therefore  a  specific  task-oriented  handbook  was 
developed  in  the  project. 

The  aim  of  the  heuristic  analysis  was  the  detection  and  exact  description  of  all  existing 
errors  of  interface  design  and  the  interaction  of  all  components  of  the  ADS  CWP. 

The  results  of  the  heuristic  analysis  of  the  ADS  CWP  show  a  lot  of  interesting  faults  of 
design,  which  can  help  redesigning  the  interface  more  user  friendly.  They  do  not  call  the 
ADS  CWP  into  question  but  they  are  quite  good  suggestions. 

Presentation  of  selected  results 

The  subjects  could  work  very  quickly  with  the  ADS  CWP  in  an  correct  manner.  They  did 
not  take  long  to  learn  the  system  operations.  This  observation  points  to  a  compatible, 
consistent,  and  clear  design  of  the  ADS  CWP  interface.  Although,  a  further  experimental 
study  to  check  the  possibility  of  working  on  job-tasks  in  a  justifiable  time  and  with  less 
mistakes,  has  to  be  conducted. 

The  working  on  particular  tasks  was  personally  good  valued  by  the  subjects.  The  errors 
of  design  for  instance  overlapping  of  menus,  to  small  types,  and  missed  flexibility  in 
colour  design  limit  the  fulfilling  of  tasks. 

Quite  interesting  in  the  results  was  the  different  judgement  of  working  with  the  ADS 
CWP  by  both  subject  groups.  For  example,  the  tasks  „Detection  of  potential  conflicts“ 
and  „Resolution  of  potential  conflicts**  were  judged  easier  to  solve  by  the  experts  of  air 
traffic  control.  The  experts  of  interface  design  had  have  a  lot  of  problems.  The  following 
figure  (Fig.  2.1 7,  Page  13)  will  show  these  results  more  clearly. 
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Take  over  of  CWP 
AcfusI  configuration  via  Seft4>-Briding 
Ac^usf  configuralion  via  rrenues 
Observing  traffic  on  radar  screen 
Observing  traffic  in  InfoBox 
Observe' whole  traffic  situation 
Takeo\'er/handoverofAC 
D^ection  of  potential  conflicts 
Resolution  of  pot.  conflicts 
.  HandcverofCWP 


■  Median  1  2  3  4  5 


Figure  0-1:  Evaluation  of  the  quality  of  working  with  ADS  CWP  (experts  of  interface  design);  Evaluation 
standard:  „The  task  is  possible  to  execute  in  a  good  manner  with  tlie  functionality  of  ADS 
CWP.“;  l=is  absolutely  incorrect,  to,  5=is  absolutely  correct. 


ffi  Median  1  2  3  4  5 


Figure  0-2:  Evaluation  of  tlie  quality  of  working  with  ADS  CWP  (experts  of  air  traffic  control); 
Evaluation  standard:  ,,Thc  task  is  possible  to  execute  in  a  good  manner  with  the  functionality  of  ADS 
CWP.“;  l=is  absolutely  incorrect,  to,  5=is  absolutely  correct. 

A  possible  reason  for  the  differences  in  evaluation  is  the  amount  of  experience  with  the 
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research  topic,  the  air  traffic  control.  To  execute  the  given  tasks  it  is  necessary  to  use  a 
clear  interface,  but  also  to  have  knowledge  about  the  priorities  of  information  and  their 
meanings  for  instance  in  the  solving  process  of  conflicts.  Which  traffic  constellation  or 
information  of  the  flight  plans  are  indicators  of  potential  conflicts?  The  controllers  could 
use  the  showed  information  on  screen  in  a  good  manner.  Experts  of  interface  design  had 
much  more  problems  with  very  complex  scenarios  compared  with  controllers.  They  could 
not  control  the  whole  traffic  situation  and  could  not  distinguish  between  topical 
important  information. 

The  obtained  results  confirm  the  splitting  of  subjects  and  show  the  importance  of 
evaluation  of  such  a  complex  interface  by  experts  of  interface  design  and  users  in  future. 

Conclusion 

In  the  context  of  further  research  the  ADS  CWP  will  be  completed  by  additional  ADS 
events  e.g.  'flight  plan  conformance  monitoring',  and  'clearance  validation'  as  well  as 
the  revision  of  design  lacks. 

The  modified  ADS  CWP  will  be  the  experimental  system  of  further  examinations  of 
ATC  operations  using  ADS.  Therefore,  a  future  structure  of  ATC  sectors  based  on 
PHARE  (Program  for  Harmonization  of  ATM  Research  in  EUROCONTROL)  has  to  be 
implemented.  Furthermore,  the  definition  of  special  ADS  trails  will  permit  an  efficient 
exploitation  of  the  airspace.  Main  emphases  will  be  the  examination  of  transition 
scenarios  between  ADS  trails  and  standard  routes  as  well  as  the  impact  on  operational 
procedures.  Changes  possible  in  controller  team  work  between  CC  and  EC  and  their 
specific  tasks  require  also  additional  research.  Increasing  quality  of  data  given  by  ADS 
win  propably  enable  an  optimization  of  long  term  planning  functions  for  CC. 
Furthermore,  the  impact  of  automated  assistance  e.g.  for  conflict  management  has  to  be 
analysed. 

A  further  simulation  phase  will  be  the  base  for  the  experimental,  comparative  analysis  of 
modifications  mentioned  above.  The  close  co-operation  with  active  controllers  of  the 
ACC  Berlin  will  guarantee  the  user  oriented  approach.  Finally,  after  the  evaluation  of  the 
modified  ADS  CWP  a  styleguide  has  to  be  developed  for  controUer  work  position  using 
ADS. 
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Abstract 

Diadem,  created  by  THOMSON-CSF,  is  a  methodology  for  specifying  and  developing 
user  interfaces.  It  improves  productivity  of  the  interface  development  process  as  well  as 
quality  of  the  interface.  The  method  provides  support  to  user  interface  development  in 
three  aspects,  (1)  Diadem  defines  roles  of  people  involved  and  their  tasks  and  organises 
the  sequence  of  activities.  (2)  It  provides  graphical  formalisms  supporting  information 
exchange  between  people.  (3)  It  offers  a  basic  set  of  rules  for  optimum  human-machine 
interfaces.  The  use  of  DIADEM  in  three  areas  (process  control,  sales  support,  and  multi- 
media  presentation)  was  observed  and  evaluated  by  our  laboratory  in  the  European  proj¬ 
ect  DIAMANTA  (ESPRIT  P20507),  The  method  provides  an  open  procedure  that  leaves 
room  for  adaptation  to  a  specific  application  and  environment.  This  paper  gives  an  over¬ 
view  of  Diadem  and  shows  how  to  extend  formalisms  for  developing  multimedia  inter¬ 
faces. 

Introduction 

From  November  1995  to  December  1996,  the  Systems  Engineering  and  Human-Machine 
Systems  Laboratory  at  the  University  of  Kassel  participated  in  the  project  DIAMANTA, 
supported  by  the  European  Union  as  project  20507  in  the  ESPRIT  programme.  Goal  of 
this  project  was  to  confirm  the  suitability  of  the  software  development  methodology 
Diadem  (Dialogue  Architecture  and  Design  Method)  as  a  means  of  developing  graphical 
user  interfaces  that  inherently  satisfy  user  needs.  Our  group  had  a  two-fold  role.  First,  we 
provided  one  of  the  three  teams  that  experimentally  applied  DiadEM  to  a  software  devel¬ 
opment  problem.  Second  and  independently  fi*om  the  first,  we  designed  and  applied  a 
formal  method  to  observe  the  three  teams,  to  determine  their  progress  and  experiences 
and,  thus,  to  assess  the  benefits  of  using  Diadem  itself.  We  write  this  paper  from  the 
views  of  the  manager  (a  D I  AD  EM  role  described  later)  of  the  development  team  as  well  as 
from  the  view  of  the  main  responsible  person  for  the  evaluation. 

The  other  partners  in  DIAMANTA  were  THOMSON-CSF  (France),  Sistemas  y 
Tratamiento  de  Informacion  S.  A.  (STI,  Spain),  and  Informationssysteme  fur  computer- 
integrierte  Automatisierung  GmbH  (ISA,  Germany),  THOMSON-CSF  provided  knowl¬ 
edge  about  Diadem  and  supervised  its  application. 

We  will  briefly  outline  the  methodology  itself  and  report  our  experiences  gained  during 
application  of  the  methodology  to  the  development  of  a  process  control  interface.  We  will 
show  how  to  extend  D I  AD  EM  to  different  and  new  applications  and  figure  out  how  these 
adaptations  may  look  like  for  multimedia  applications  in  process  control. 
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The  Diadem  Methodology 

K  process  of  user  interface  development  becomes  more  and 

more  an  important  economic  factor  for  software  development 
companies.  Experience  shows  that  the  user  interface  software  re¬ 
quires  about  20%  to  50%  of  the  total  software  development  costs. 
On  the  other  hand,  a  well- designed  user  interface  improves  the  usability  of  the  product 
and,  by  this,  the  efficiency  of  its  use  and  its  acceptance  by  the  users. 

Against  this  background,  the  Diadem  methodology  aims  at  giving  software  developers 
support  for  their  development  activities.  Based  on  a  dialogue  model  and  an  ergonomic 
approach  to  user  interface  design,  THOMSON-CSF  COMMUNICATIONS  created 
Diadem  in  1992  in  conformation  to  the  THOMSON-CSF  software  reference  system. 
From  the  beginning,  the  method  featured  an  adaptable  multi-modal  dialogue  based  on  er¬ 
gonomic  principles.  Use  in  over  30  projects  in  the  THOMSON  group  provided  feedback 
for  a  new  version,  until  from  the  end  of  1995  the  project  DIAMANTA  evaluated 
Diadem. 

The  objectives  of  Diadem  are  to  enhance  the  efficiency  in  developing  human-machine 
interfaces  and  to  ensure  the  quality  of  the  user  interfaces  to  be  developed.  These  objec¬ 
tives  shall  be  achieved  by  providing  a  guide  to  structure,  organise,  and  supervise  all  ac¬ 
tivities  of  the  software  development  team.  For  the  activities,  D I  AD  EM  defines  develop¬ 
ment  phases,  for  the  team  members,  D I  AD  EM  defines  roles.  We  will  describe  both  in  the 
following  sections  along  with  our  experience  made  during  the  DIAMANTA  project. 

Diadem  Roles 

Diadem  suggests  distinguishing  the  role  of  a  manager,  a  specifier,  a  human  factors  spe¬ 
cialist,  a  programmer,  and  a  user.  It  was  a  large  industrial  company  that  developed 
Diadem,  and,  therefore,  the  original  users  of  the  method  were  large  teams.  All  teams  in 
our  project  were  smaller.  The  practical  evaluation  of  Dl  AD  EM  in  our  group  as  well  as  in 
the  other  development  teams  showed  that  it  is  difficult  to  have  exactly  these  persons  or 
group  of  persons  covering  these  professions  constantly  available  in  a  development  team. 
Thus,  it  is  better  to  speak  of  roles  covered  by  different  people  in  the  development  team 
while  one  person  may  cover  different  roles  as  well  as  one  role  may  keep  several  persons 
busy. 

By  giving  roles,  Diadem  describes  the  responsibilities  and  activities  of  the  people  in¬ 
volved  in  the  software  development.  The  manager  is  responsible  for  the  overall  organisa¬ 
tion  and  supervision  of  the  development  process.  The  specifier  interacts  closely  with  the 
end  user  during  task  analyses  and  develops  comprehensive  specifications  for  the  user  in¬ 
terfaces.  These  specifications  formally  describe  the  interaction  of  the  user  with  the  inter¬ 
face  and  build  the  basis  for  the  software  coding  activities  of  the  programmer.  It  appeared, 
that  the  original  role  of  the  human  factors  specialist  should  better  be  distributed  to  two 
roles:  The  cognitive  ergonomist  and  the  practical  ergonomist.  The  cognitive  ergonomist 
supports  the  specifier  during  the  tasks  analyses  and  ensures  an  appropriate  breakdown  of 
the  user  activities.  The  practical  ergonomist  generates  the  Man-Machine  Interface  (MMI) 
Handbook.  This  is  an  application- specific  set  of  rules  ensuring  consistency  and  usability 
of  the  interface,  based  on  a  general  set  of  ergonomic  rules  and  guidelines.  In  addition  to 
the  task  analyses  at  the  beginning  of  the  development,  the  ergonomists  involve  end  users 
throughout  the  whole  development  process  in  usability  tests. 
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The  development  team  at  the  University  of  Kassel  involved  five  persons  on  the  develop¬ 
ment  side  and  two  end  users.  One  student  acted  as  specifier  while  writing  his  diploma 
thesis  on  task  breakdown  and  operation  strategies  in  the  apphcation.  Two  students 
worked  as  practical  ergonomists,  writing  homework  (Studienarbeit)  on  human  factors 
guidelines  in  the  application-specific  MMI  Handbook.  The  programmer  was  a  mechani¬ 
cal  engineer  who  just  finished  his  degree.  An  engineer  and  a  senior  operator  from  a 
chemical  plant  supported  the  development  team  as  end  users  of  the  system  to  be  devel¬ 
oped.  Finally,  one  of  the  authors  covered  the  roles  of  manager,  human  factors  specialist, 
and  partly  of  the  programmer,  and  he  also  assisted  the  specifier.  This  last  combination  of 
roles  in  one  person  contradicts  some  of  the  basic  ideas  of  D I  AD  EM,  in  which  the  human 
factors  specialist  should  defend  the  user’s  needs  against  the  manager.  However,  the  two 
practical  ergonomists  counterbalanced  this  inadequacy  during  enthusiastic  debates. 

Diadem  Formalisms 

In  the  whole,  D I  AD  EM  focuses  on  having  a  common  basis  for  conmiunication  within  the 
development  team,  and  on  being  able  to  supervise  and  trace  the  development  process. 
More  than  that,  the  comprehensive  formal  descriptions  of  the  user  interface  make  mainte¬ 
nance  and  further  development  of  the  software  product  easier.  To  co-ordinate  the  work 
within  the  development  team,  DIADEM  provides  several  formal  specifications.  A  task 
graph  represents  a  first  formal  description  of  the  users'  task  derived  from  a  task  analysis. 


The  task  graph  shows  a  breakdown  of  the  user’s  goals  in  a  hierarchical  structure.  So- 
called  use  relations  connect  the  levels  of  the  hierarchy.  To  achieve  the  specific  goal,  the 
user  uses  the  tasks  on  the  lower  level.  The  task  graph  in  our  application  showed  the  over¬ 
all  goal  "produce  product"  and  lower-level  goals  like  "start  die  system"  and  "correct  er¬ 
ror".  Figure  1  shows  a  more  everyday  example. 

The  task  graph  enforces  re-use  of  software  components:  On  the  lower  levels,  a  small 
number  of  generic  tasks  were  used  to  reach  several  goals  on  a  higher  level.  In  the  process 
control  application,  the  two  tasks  "change  a  pump  status"  and  "set  a  valve  position"  alone 
could  form  the  lowest  level.  For  the  implementation,  this  means  only  two  software  mod¬ 
ules,  one  to  control  a  pump  and  another  for  operating  a  valve. 
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Further  formalisms  provided  by  D I  AD  EM  are  technical  sheets,  one  for  each  task  in  the 
task  graph.  The  technical  sheet  comprises  a  strategy  graph,  giving  an  abstract  description 
of  the  interaction  between  user  and  dialogue  system.  The  strategy  graphs  formed  the  main 
source  of  information  for  the  programmer.  Using  a  window-based  system,  the  program¬ 
mer  created  one  dialogue  window  for  each  task.  The  strategy  defined  the  logic  of  interac¬ 
tions  in  this  window.  It  is  important  to  note  that  the  strategies  only  define  the  logic  of  in¬ 
teractions.  They  do  not  define  the  mechanism  and  media  to  be  used.  To  keep  use  of 
mechanism  and  media  consistent  throughout  the  development,  they  are  defined  in  the  ap¬ 
plication’s  MMI  Handbook. 

This  MMI  Handbook  is  another  formalism  provided  by  Diadem.  With  the  specifications 
recorded  here  the  programmer  can  turn  the  strategies  into  algorithms  driving  the  interface 
in  a  correct,  economical  sound  and  consistent  way.  Rules  defining  how  to  implement  an 
interaction  in  a  specific  application  enforce  consistency.  This  starts  with  common  pres¬ 
entation  choices  like  colours  of  window  border,  window  background,  character  fonts 
used,  and  it  continues  with  window  organisation  and  failure  handling.  The  MMI  Hand¬ 
book  must  cover  the  whole  range  of  decisions  to  take  when  implementing  the  strategies. 
If  not  -  it  needs  to  be  extended  with  the  necessary  additional  decisions.  Figure  2  shows 
rules  from  the  handbook  written  for  our  application. 
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Figure  2  Definitions  in  the  MMI  Handbook  (from  Engel  and,  Quittkat,  1996) 


The  Diadem  handbook  "Guidelines  for  Authors"  (THOMSON  1995,  4)  provides  a  table 
of  topics  to  be  treated  in  the  MMI  Handbook.  A  set  of  general  rules  is  contained  in  a  ge¬ 
neric  MMI  Handbook  (THOMSON,  1995,  5)  that  can  be  extended  and  adapted  to  any 
kind  of  application  to  ensure  the  consistency  of  the  user  interface  and  the  adherence  to 
ergonomics  rules. 
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Diadem  Activities 

Besides  proposing  a  certain  staff  composition  for  the  development  team  and  providing 
formalisms  to  specify  the  user  interface,  Diadem  also  suggests  how  to  organise  the  de¬ 
velopment  process.  In  the  ideal  case,  this  process  should  follow  the  downward  part  of  the 
V-model,  covering  system  specification,  software  specification,  preliminaiy  design,  de¬ 
tailed  design,  iterative  modification,  and  finishing  phase.  Nevertheless,  even  if  the  devel¬ 
opment  process  will  not  follow  the  V-model,  this  will  not  affect  the  possibility  to  make 
use  of  the  concepts  and  formalisms  provided  by  the  method.  D I  AD  EM  defines  activities 
grouped  in  four  development  phases,  called  System  Requirements,  Software  Require¬ 
ments,  Preliminary  Design,  and  Detailed  Design, 

The  System  Requirements  phase  starts  with  a  task  analysis  and  defines  requirements  and 
capabilities  of  the  user  interface  based  on  user  goals  and  means  necessary  to  achieve 
these  goals.  The  Software  Requirements  phase  defines  descriptions  of  the  user’s  tasks  and 
user  objects.  The  MMI  Handbook  is  written  during  this  phase,  defining  the  future  appear¬ 
ance  of  the  interface  or,  at  least,  of  the  first  prototype.  Diadem  enforces  the  early  genera¬ 
tion  and  evaluation  of  prototypes  involving  end  users,  leading  to  modifications  of  the 
definitions  in  several  cycles.  TTie  graphical  formalisms  described  above  ease  user  com¬ 
ments  to  software  designers.  The  software  architecture  is  designed  during  the  Preliminary 
Design  phase  while  in  the  Detailed  Design  phase  presentation  and  dialogue  descriptions 
are  completed. 

Our  project  covered  a  development  process  up  to  the  preliminary  design  phase  involving 
four  prototypes  of  growing  complexity  and  functionality.  As  with  the  roles,  here  again  the 
problem  of  a  small  group  compared  to  a  large  software  development  company  shows  up. 
It  was  not  possible  to  follow  strictly  the  sequence  of  development  phases,  as  all  the  peo¬ 
ple  in  the  development  team  must  be  kept  continuously  busy  in  the  assigned  roles 
throughout  the  project.  Thus,  phases  were  overlapping  and  not  consecutive  and  activities 
were  scheduled  more  regarding  the  availability  of  people  than  depending  on  the  com¬ 
pleteness  of  required  inputs.  For  example,  MMI  prototyping  started  when  the  program¬ 
mer  was  available  and  the  first  user  task  descriptions  existed;  it  did  not  wait  for  a  com¬ 
plete  task  analysis  and  a  satisfactory  MMI  requirements  definition.  However,  going 
through  four  prototyping  cycles  in  four  months  step-wise  compensated  the  early  deficien¬ 
cies. 

Evaluation  of  DiadeM 

The  Diadem  methodology  for  user  interface  development  has  been  designed  to  be  appli¬ 
cable  to  any  kind  of  software  apphcation.  The  Trial  Application  Project  DIAMANTA 
was  initiated  to  show  that  this  goal  could  be  achieved.  Therefore,  Diadem  was  used  to 
develop  three  very  different  kinds  of  software  applications.  These  three  applications  were 
a  tourism  information  system,  developed  by  STI;  a  sales  support  system  for  used  cars, 
developed  by  ISA;  and  an  interface  to  chemical  process  control,  developed  by  our  group. 

By  applying  D I  AD  EM  to  develop  the  applications,  its  overall  suitability  was  evaluated  and 
the  specific  costs  and  benefits  of  using  DIADEM  were  pointed  out.  This  was  done  by 
means  of  an  evaluation  procedure  especially  designed  for  the  DIAMANTA  project. 

The  primary  focus  of  the  DIAMANTA  evaluation  activities  was  on  the  efficiency  of  the 
software  development  processes.  However,  the  quality  and  usability  of  the  user  interfaces 
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to  be  developed  were  a  second  important  aspect.  Therefore,  we  designed  an  evaluation 
and  measurement  procedure  to  cover  these  two  criteria.  This  procedure,  as  shown  in 

Figure  3,  is  based  on  three  parts:  Continual  Evaluations,  Periodical  Evaluations,  and  Fi¬ 
nal  Evaluations. 


Figure  3:  The  DIAMANTA  evaluation  procedure 

Weekly  Continual  Evaluations  accompanied  the  development  processes.  They  recorded 
the  effort  for  development  activities  and  subjective  assessments  by  the  developers  in 
regular  intervals.  Periodical  Evaluations  followed  each  development  phase  completed. 
These  aimed  at  recording  the  current  state  of  the  development  process  as  well  as  the  as¬ 
sessments  by  the  developers  referring  to  the  respective  development  phase.  Furthermore, 
we  demanded  to  perform  evaluations  of  the  specifications  and  the  interface  prototypes  in 
co-operation  with  real  end  users  after  finishing  a  development  phase.  The  evaluation  pro¬ 
cess  ended  with  a  Final  Evaluation.  It  consisted  of  two  parts:  concluding  assessments  of 
the  developers  referring  to  the  whole  development  process  and  a  usability  test  of  the  de¬ 
veloped  interface  by  the  end  user. 

All  data  recorded  along  the  DIAMANTA  trial  application  developments  were  processed, 
analysed  and  interpreted.  Finally,  the  following  conclusions  were  drawn: 

0  Diadem  is  sufficiently  generic  and  flexible  to  use  it  in  a  large  range  of  appli¬ 
cations  and  organisations, 

1  The  adaptation  of  D I  AD  EM  to  the  specific  projects  would  have  been  much 
easier  with  the  Customisation  Guide  now  available. 

2  The  generic  MMI  Handbook  provided  by  Diadem  is  a  valuable  basis  to  de¬ 
velop  a  consistent  interface. 

3  The  formalisms  provided  by  D I  AD  EM  turned  out  to  be  a  good  basis  for  com¬ 
munication  within  the  development  team.  They  represent  unambiguous  speci¬ 
fications  for  the  user  interface,  improve  the  efficiency  of  the  development 
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process,  and  make  later  modifications  or  extensions  easier. 

4  The  dialogue  specifications  are  to  a  high  degree  independent  from  the  kind  of 
interface,  thus,  it  will  be  possible  to  re-use  existing  specifications  as  the  basis 
for  development  of  future  interfaces,  including  interfaces  which  use  emerging 
interaction  technologies. 

As  pointed  out  above,  the  evaluation  of  the  Diadem  methodology  within  the  DIA- 
MANTA  project  led  to  very  satisfactory  results.  DIADEM  increased  the  efficiency  of  all 
three  application  developments,  and  the  user  interfaces  developed  were  of  a  good  quality 
and  usability  by  the  end  users  of  the  respective  systems. 

The  good  overall  result  is  based  on  a  comprehensive  definition  of  user  requirements  at 
the  very  beginning  of  the  development  activities  and  on  a  constant  involvement  of  end 
users  in  the  development  process.  This  leads  to  a  reduced  effort  for  expensive  changes  at 
a  later  stage. 

Furthermore,  all  requirements  identified  and  all  specifications  become  described  in  a 
formal  way.  This  improves  tracing  the  development  process  and  communication  between 
the  people  involved.  Even  validating  the  software  developed  is  simplified  by  the  formal 
descriptions.  Finally,  these  specifications  are  invaluable  means  for  developing  future  ver¬ 
sions  of  the  software. 

One  significant  result  of  the  evaluations  performed  in  the  framework  of  the  DIAMANTA 
project  is  that  the  evaluations  of  the  three  very  different  applications  led  to  the  same  con¬ 
clusions.  Additionally,  evaluation  results  of  the  development  processes  correspond  with 
the  results  of  a  heuristic  evaluation  of  Diadem  from  a  theoretical  point  of  view.  This 
suggests  that  the  conclusions  drawn  are  valid  even  for  a  broad  field  of  applications. 

Customising  Diadem  to  Multimedia  Interfaces 

The  common  benefits  of  using  Diadem,  as  pointed  out  in  the  DIAMANTA  project,  are  a 
result  of  the  generic  concept  of  the  D I  AD  EM  methodology.  However,  this  generality  also 
induces  the  need  to  customise  the  method  to  a  specific  application. 

Multimedia  applications  are  a  new  field,  providing  features  that  are  quite  different  from 
those  of  conventional  software.  Therefore,  developing  multimedia  interfaces  puts  new  re¬ 
quirements  on  the  specification  work.  To  meet  these  new  requirements,  the  formalisms 
provided  by  DIADEM  must  be  extended  in  an  appropriate  way.  The  significance  of  mul¬ 
timedia  extensions  to  process  control  interfaces  is  already  stressed  in  another  contribution 
from  our  group  in  this  conference  (Borys  and  Johannsen,  1997),  Backed  by  our  good  ex¬ 
perience  during  the  trial  application  of  Diadem,  we  now  consider  continuing  the  use  of 
the  skill  gained  in  future  software  development  projects. 

The  main  anchor  point  we  see  for  extending  Diadem  to  multimedia  applications  is  the 
MMI  Handbook.  Task  analyses  deliver  task  graphs  and  strategy  drawings  describing  dy¬ 
namics  of  interactions  necessary  to  operate  an  application.  They  leave  completely  open 
by  what  means  or  media  this  interaction  would  take  place.  The  MMI  Handbook  is  based 
on  the  ’’General  Rules  and  Guidelines”  (THOMSON  1995,  5).  First,  this  handbook  needs 
extensions  to  cover  the  decision  ’’what  medium  to  use  for  what  type  of  interaction".  A  be¬ 
ginning  is  already  made  in  Chapter  5  Dialogue:  Common  Choices  with  rule  DIAMOl: 
"The  dialogue  modes  must  be  chosen  taking  into  account  the  characteristics  of  the 
users  and  the  task  requirements:  For  example  ...  use  frequency,  ...  time  con- 
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straints..."  (THOMSON  1995,  5,  pg.  59). 

Some  new  basic  rules  should  cover  ergonomics  of  additional  media:  Size  of  video  pres¬ 
entation,  level  of  acoustical  output,  speed  of  synthesised  speech.  An  interesting  research 
topic  would  relate  to  spatial  sound  sources,  corresponding  to  Dl  AD  EM  rules  in  Section  4.6 
of  the  handbook  covering  arrangement  of  elements  on  the  screen.  Rule  SPGR03  defines 
the  greatest  distance  for  grouped  objects,  SPGR04  the  minimum  distance  for  separated 
objects,  both  in  visual  angel.  Besides  the  angle,  a  rule  for  sound  presentation  must  take 
care  of  differences  in  frequency,  perceived  distance,  and  loudness. 

Some  amount  of  new  rules  will  certainly  extend  Chapter  4  ("Presentation:  Common 
Choices")  and  Chapter  9  ("Interaction  Tools"),  Especially  the  latter,  which  currently  only 
mentions  cursor  and  mouse,  will  undergo  some  extensions.  It  must  cover  additional  input 
media  like  voice  or  the  famous  (but  expensive)  cyber  glove,  but  it  also  must  regard  addi¬ 
tional  more-dimensional  mouse  operations  like  pointing  in  space  and  rotations.  Rules 
could  be 

"when  dialogue  mode  is  question/answer  and  presentation  mode  is  speech  and  an¬ 
swer  is  short,  input  medium  should  be  voice" 
or 

"when  object  moves,  video  bandwidth  must  permit  a  frame  rate  of. . . 

Finally,  our  aim  is  always  to  involve  the  user  in  the  development.  Thus,  a  basic  guideline 
would  be 

"when  the  user  wants  to  hear  it,  provide  sound,  when  the  operator  is  used  to  feel  it, 
provide  vibrations". 

This  will  bring  back  the  operator  from  the  distant  control  room  to  the  plant  again,  as  it  is 
also  required  in  our  other  contribution  to  this  conference. 

Conclusions 

Diadem,  a  methodology  for  development  of  user  interfaces,  leads  developers  through  the 
developing  process.  One  of  the  last  steps  is  turning  strategies,  derived  from  task  analyses, 
into  program  code.  When  extending  D I  AD  EM  to  multimedia  interfaces,  this  last  step 
needs  to  take  into  account  the  capabilities  of  new  media  available.  Additional  research, 
extending  the  mle  set  provided  by  D I  AD  EM,  will  provide  a  useful  basis  to  develop  multi- 
media  interfaces. 
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What  is  Multimedia? 


Proposal  of  a  Taxonomy  for  Human-Machine  Communication 

Georg  Geiser 
Katholische  Universitdt  Eichstatt 
D-85071  Eichstatt,  Germany 


Abstract 

That  there  is  no  accurate  or  standard  definition  of  multimedia  is  a  firequent  statement  in 
publications  on  this  subject.  After  a  survey  on  the  resulting  increasing  variety  of  defini¬ 
tions,  the  existing  terminology  is  characterised  by  discrepancy,  inconsistency,  incom¬ 
pleteness,  and  lack  of  task  orientation.  A  proposal  of  a  taxonomy  for  multimedia  is 
developed  with  three  categories:  medium,  representation  and  modality,  which  are  part  of 
a  three-level  concept  being  applicable  to  the  input,  and  the  output  side  of  the  human  user 
as  well.  Thus,  a  clear  separation  of  technical  devices,  modes  of  information 
representation,  and  of  human  sensory  and  effector  channels  is  achieved.  The  importance 
of  a  task-related  design  of  multimedia  is  finally  shown  by  means  of  two  examples  which 
illustrate  the  relationship  of  the  taxonomy  to  multimedia  design  tasks. 

Structure  and  Development  of  Human-Computer  Interaction 

Before  analysing  multimedia  human-computer  interaction,  the  essential  input  and  output 
channels  of  human  information  processing  are  briefly  described.  Figure  1  shows 
examples  of  the  multimodality  of  human  information  input  and  output,  i.  e.  there  are 
multiple  modalities  for  receiving  and  sending  information.  Sensoiy  modalities  on  the 
input  side  are  the  classical  five  senses,  however,  actually  only  the  eye,  the  ear,  and  the 
tactile  sense  are  suitable  for  human-machine  communication.  For  the  output  effector 
modalities  such  as  mimic,  voice,  and  gesture  are  used. 


Figure  1.  Human  information  input  and  output  channels. 
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Figure  2.  Development  of  human-machine  communication. 


These  means  given  by  human  nature  have  to  be  seen  in  relation  to  the  technologies  for 
presentation,  processing,  and  input  of  information.  It  is  interesting  to  look  at  the  history 
of  the  corresponding  technical  devices.  In  Figure  2,  important  steps  of  the  evolution  of 
human-machine  communication  are  presented.  First,  digits  were  the  coding  form  for 
information  to  be  exchanged  between  human  and  machine,  which  were  represented  by 
means  of  dials,  digital  displays,  and  rotary  knobs.  Then  screen  and  keyboard  allowed  to 
include  text.  Recently,  speech  and  image  are  additional  coding  forms  for  input  and 
output.  And  actually  expert  systems  are  including  knowledge  representation. 
Furthermore,  virtual  reality  allows  the  simulation  of  working  and  learning  environments 
which,  up  to  now,  were  not  available  in  reality  due  to  cost  reasons  or  dangers.  Today 
computer  networks  offer  increasing  conununication  services  for  numerous  professional 
and  private  applications. 

This  development  results  in  multimedia  technology  allowing  increased  adaptation  of  ma¬ 
chines  to  the  properties  of  human  information  processing  and  even  expanding  human  ca¬ 
pabilities.  A  multimedia  computer  is  supposed  to  process  not  only  columns  of  numbers 
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and  sequences  of  letters  as  in  the  beginning  of  computer  era  but  also  to  handle  sound, 
speech,  graphics,  and  image  sequences  as  input  and  output  material  or  as  transmitted  net¬ 
work  information. 

Various  Definitions  of  Medium  and  Multimedia 

In  various  application  areas  multimedia  moved  into  the  focus  of  interest,  e.  g.  in  market¬ 
ing,  telecommunication,  and  computer-based  learning.  Though  used  as  a  buzzword  or 
collective  noun,  there  are  numerous  efforts  to  find  a  taxonomy  for  this  new  technical 
area.  In  the  following  different  definitions  are  compared  by  deliberately  concentrating  on 
the  area  of  human-machine  communication.  Interpersonal  communication  as  the  other 
field  of  interest  represented  by  publicity,  television  or  arts  is  not  considered  here. 

Even  for  the  term  "medium"  many  definitions  can  be  found,  few  of  them  are  hsted  in 
Table  1  in  order  to  show  the  range  of  variance.  In  some  references  the  definition  of 
medium  is  omitted  (e.  g.  Interactive  Multimedia  Association  (IMA)  1996)  or  only  an 
implicit  definition  is  given  by  simply  listing  types  of  media  (e.  g.  Heller,  Martin  1995). 
Burger  1993  assigns  medium  to  a  transmission  means  in  the  sense  of  physics,  e.  g.  air, 
water.  ISO  draft  standard  ISO/IEC  CD  13522-1,  1993  (MHEG  1993)  defines  medium  in 
a  very  comprehensive  way  as  human  and  technical  means  for  perception,  representation, 
storage,  transmission,  and  exchange  of  information. 

Weidenmann  1 994  points  out  that  definitions  of  medium  are  often  vague  and  therefore 
he  specifies:  A  medium  is  a  technical  means  for  presentation  or  storage  of  information 
for  the  human  user.  Blattner  and  Ghnert  1996  agree  with  him  roughly  when  they 
designate  media  as  information  carriers.  In  their  media  taxonomy,  Heller  and  Martin 
1995  define  text,  graphics,  sound  and  motion  as  four  types  of  media  and  additionally 
they  assign  three  categories  of  expression  to  each  medium.  Khmsa  1995  uses  a  similar 
classification  of  media  types.  Additionally  he  distinguishes  between  time  independent 
(text,  graphics)  and  time  dependent  (video,  audio,  animation)  media. 


Author(s) 

Definition  of  „Medium“ 

Burger  1993 

Means  for  transmission  for  stimuli:  air,  water  etc. 

MHEG  1993 

Means  for  perception,  representation,  presentation,  storage, 
transmission  and  exchange  of  information 

Weidenmann 

1994 

Blattner,  Glinert 
1996 

Means  for  presentation  and  storage  of  information  (paper, 
screen  etc.) 

Information  carrier  (paper,  paint,  video,  CD-ROM,  screen  etc.) 

Heller,  Martin 
1995 

Klimsa  1995 

Types  of  media:  text,  graphics,  sound,  motion 

Time  independent  (text,  graphics)  and  time  dependent  media 
(video,  audio,  animation) 

Table  1.  Vaiious  definitions  of  "medium"  by  different  authors. 
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As  there  is  no  agreement  on  the  term  medium,  the  definitions  of  "multimedia"  inevitably 
must  show  at  least  a  comparable  variety,  A  first  example  is  given  by  Steinmetz  1993:  A 
multimedia  system  is  characterised  by  computer-controlled,  integrated  generation, 
manipulation,  presentation,  storage,  and  communication  of  independent  information  ele¬ 
ments  being  coded  at  least  with  one  continuous  (time  dependent)  and  one  discrete  (time 
independent)  medium.  Whereas  Koegel  Buford  1994  calls  the  simultaneous  use  of 
different  media  (voice,  video,  text,  animation  etc.)  multimedia. 

Weidenmann  1994  criticises  available  definitions  as  confounding  the  categories  techni¬ 
cal  devices,  presentation  modes,  and  sensory  channels.  He  uses  three  terms  for 
multimedia  information  presentation:  medium,  coding,  and  modality,  hi  addition  to  his 
already  mentioned  definition  of  medium,  he  declares  coding  as  symbol  systems  for 
presentation  of  information.  Following  established  definitions  he  assigns  modality  to 
human  sensory  channels.  Blattner  and  Glinert  1996  end  up  with  similar  terms,  however 
they  leave  an  ambiguity  of  modality  by  allowing  the  two  meanings  sensory  channel  and 
interaction  style. 

In  his  definition  of  multimedia,  Klimsa  1995  claims  the  use  and  integration  of  several 
media,  reference  to  the  application  and  multimodahty  as  essential  aspects.  Multimodality 
is  understood  as  parallel  processing  and  simultaneous  presentation  of  several  media  and 
interactivity,  Hofstetter  1995  points  out  four  essential  components:  computer-based  in¬ 
formation  processing,  links  that  connect  the  information,  navigation  tools,  and  ways  for 
user-specific  interaction.  He  states: , Multimedia  is  the  use  of  a  computer  to  present  and 
combine  text,  graphics,  audio,  and  video  with  links  that  let  the  user  navigate,  interact, 
create,  and  communicate.“ 

The  Interactive  Multimedia  Association  (IMA)  1996  defines  multimedia:  „Delivery  of  in¬ 
formation,  usually  via  a  personal  computer,  that  combines  different  content  formats  (text, 
graphics,  audio,  still  images,  animation,  motion  video,  etc.)  and/or  storage  media 
(magnetic  disk,  optical  disc,  video/audio  tape,  RAM).“ 

These  and  other  definitions  of  multimedia  are  characterised  by  one  or  several  of  the  fol¬ 
lowing  deficiencies: 

•  Discrepancy:  There  are  considerable  differences  between  most  terminologies, 

•  Inconsistency:  Within  some  definitions  terms  are  confounded,  e.  g.  partially 
the  definition  of  a  single  term  relates  to  technical  means  and  partially  to 
presentation  forms  and  sensory  modalities.  Furthermore,  well  established 
terms  such  as  modality  are  redefined. 

•  Incompleteness:  Only  partial  aspects  are  dealt  with,  e.  g.  networking  is  mostly 
neglected  and  the  information  input  and  the  output  side  of  the  human  user  are 
not  both  included. 

•  Lack  of  task  orientation:  The  user’s  tasks  of  information  input,  processing, 
and 

output  are  generally  not  considered. 

Multimedia  Taxonomy 

In  the  following  a  more  precise,  comprehensive  and  user  and  task  oriented  taxonomy  for 
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the  interaction  of  human  with  multimedia  devices  is  proposed.  In  order  to  obtain  such  a 
definition  human  information  input  as  well  as  information  output  and  the  user’s  task 
have  to  be  considered.  Three  terms  as  part  of  a  three-level  concept  are  used  (Table  2): 

•  Medium 

To  begin  with,  a  medium  is  a  technical  means  for  one  or  several  of  the  following 
purposes:  Presentation,  storage,  input,  processing,  transmission  of  information  for  and  by 
the  human  user  (example:  screen).  The  media  available  are  determined  by  the  state  of 
hardware  and  software  technology. 

Looking  at  the  media  only  is  not  sufficient  to  describe  formally  the  information  exchange 
between  human  and  machine;  two  additional  terms  have  to  be  introduced,  representation 
and  modality. 

•  Representation 

Representation  is  the  appearance  of  information,  enabling  human  perception,  processing 
or  output  of  information  (examples:  text,  displayed  visually  or  spoken). 

•  Modality 

Modality  encompasses  human  sensor  or  effector  channels  for  input  or  output  of  informa¬ 
tion  (examples:  visual  system,  gesture). 

On  the  first  level  of  this  approach  technical  requirements  are  addressed  by  medium  and 
by  the  next  two  levels  representation  and  modality,  mainly  the  areas  of  human 
perception,  cognition,  and  motoric  are  concerned. 


Level 

Definition 

Areas  concerned 

Medium 

Means  for  presen¬ 
tation,  storage,  in¬ 
put,  processing 
and  transmission 
of  information 

Screen,  speaker, 
CD-ROM,  network 

Hardware, 

Software 

r 

Representation 

Appearance  of 
information 

Digit,  text,  colour, 

icon,  table,  bar 
graph,  hypertext, 
tone,  speech,  vir¬ 
tual  reality 

k. 

Perception, 
cognition,  and 
motor  systems  of 
human  user 

r 

Modality 

Sensor  or  effector 
channel  for  input 
and  output  of  in¬ 
formation 

Visual,  auditory, 
and  tactile  system; 
mimic,  gesture,  and 
voice 

Table  2.  Definition  of  medium,  representation,  and  modality. 


As  Table  3  shows  these  three  categories  are  apphcable  to  the  human  information  input 
and  output  as  well.  Examples  listed  for  illustration  in  Table  3  are  not  exhaustive.  As  an 
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important  aspect  the  task  of  the  human  has  to  be  mentioned  together  with  its  result 
because  use  and  therefore  the  design  of  these  2x3  levels  depend  strongly  on  the  kind  of 
task  to  be  accomplished  by  the  human  user. 

These  2x3  levels  may  be  used  in  a  restrictive  manner  by  using  minimum  numbers  of 
media,  representation  forms,  and  modality  channels.  Multidimensional  use  offers 
abundant  variety  of  interaction  by  multimedia  and/or  multirepresentation  and/or 
multimodality.  However,  these  three  levels  are  not  independent,  e.  g.  the  representation 
image  can  be  displayed  with  the  media  paper  or  screen,  but  can  only  be  perceived  with 
the  modality  eye. 


Processing  Levels  Examples 

Steps 


Input  of 

Information 

(Sensors) 

Medium 

Paper,  screen,  speaker 

Representation 

Text;  table 

Modality  (Sensors) 

Eye,  ear,  tactile  sense 

Processing 

of 

Modality  and  representation  specific 

information 

processing 

Output  of 

Information 

(Effectors) 

Modality 

Gesture,  voice,  mimic 

Representation 

Text;  formal  language 

Medium  (Effectors) 

Paper,  key,  microphone 

Table  3.  Formal  description  of  human  information  input  and  output  by  means  of  me¬ 
dium,  representation,  and  modality. 


Representation 
of  Information 

Definition 

Examples 

Coding 

System  of  symbols  for  single  information 
elements 

Digit,  text 

Organisation 

Structure  for  local,  temporal,  and  se¬ 
mantic  arrangement  of  several  information 
elements 

Table,  hypertext 

Table  4.  Subcategories  of  representation  of  information:  coding  and  organisation. 
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The  level  representation  can  be  divided  in  two  subcategories  (Table  4): 

•  Coding 

Coding  means  the  use  of  a  system  of  symbols  for  single  information  elements,  e.  g.  visual 
display  of  a  measured  variable  such  as  time  by  digits  (digital  code)  or  by  geometric  pa¬ 
rameters  (analog  code).  If  there  is  a  single  information  element  to  be  presented  merely 
coding  is  necessary,  in  the  case  of  a  set  of  information  elements  an  additional  mode  of 
representation  has  to  selected,  namely 

•  Organisation 

Organisation  of  information  describes  the  representation  of  several  information  elements, 
e.  g.  arrangement  of  several  numeric  values  by  means  of  a  table,  bargraph,  etc.  Here  the 
presentation  of  local,  temporal  and  semantic  structures  are  to  be  distinguished.  Hypertext 
is  an  example  for  the  last  type  of  organisation  allowing  reading  a  text  in  a  non-linear 
manner  according  to  the  contents. 

Thus,  human-machine  communication  is  not  only  determined  by  media  but  also  by 
representation  and  modality.  We  need  a  three-level  model  in  order  to  realise  the 
extraordinary  abundance  of  communication  modes  which  the  usual  multimedia  term  tries 
to  outline  with  a  single  notion.  Also  for  a  systematic  design  of  human-machine 
communication  this  more  detailed  terminology  seems  to  be  necessary.  So  the  following 
definition  of  multimedia  is  proposed: 

Utilisation  of  media,  L  e.  several  technical  means  for  presentation,  storage,  input,  proc¬ 
essing,  transmission  of  information,  for  the  human  user  in  order  to  permit  multiple 
representation  (coding  and  organisation)  of  the  information  which  meets  the 
requirements  of  the  human  given  by  his  modalities  (sensors  and  effectors),  cognition, 
and  tasks. 

The  proposed  terminology  is  more  comprehensive  than  the  existing  ones  by  including 
human  information  input,  processing,  and  output  as  well.  New  developments  can  be 
easily  included:  new  media  such  as  computer  networks,  new  organisation  concepts  such 
as  hypertext  or  new  interaction  modes  such  as  virtual  reality. 

Figure  3  shows  the  components  of  today’s  multimedia  human-machine  system.  Sensor 
and  effector  modalities  on  the  one  hand,  and  displays  (visual,  auditory,  tactile)  and  con¬ 
trols  (manual,  vocal)  on  the  other  hand,  build  the  interface  between  human  and  machine. 
With  these  devices  information  with  different  forms  of  coding  and  organisation  can  be 
exchanged.  Network  connections  and  other  devices  like  image  scanners  make  additional 
information  available. 

Examples 

In  the  following  the  application  of  the  proposed  multimedia  taxonomy  is  shown  by 
means  of  two  examples.  In  the  first  application  (Table  5),  which  is  based  on  the  task  to 
acquire  ornithological  knowledge,  only  the  part  of  information  presentation  for  the 
learner  is  considered.  The  second  case  concentrates  on  information  output  while  learning 
a  foreign  language  (Table  6),  These  simple  examples  illustrate  the  essential  multimedia 
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design  tasks  to  be  accomplished:  choice  of  media,  coding  and  organisation  of 
information  based  on  human  properties  of  information  input,  cognition,  and  output  in  a 
task  oriented  manner. 


Figure  3.  Multimedia  human-machine  system. 


Task 

Acquisition  of 
ornithological 
knowledge 

Medium 

Repres< 

Coding 

sntation 

Organisation 

Modality 

(Sensor) 

Name,  family 

Dissemination 

Appearance 

Flight  behaviour 
Song 

Screen 

Text 

Graphics 

Image 

Image  sequence 
Analog  function 

Table 

Diagram 

Legend 

Time  scale 
Sonogram 

Eye 

Actual  research 

Network 

Text 

Hypertext 

Song 

Loudspeaker 

Sound 

- 

Ear 

Table  5.  Example  for  the  application  of  the  multimedia  taxonomy  for  human  informa¬ 
tion  input. 
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Task 

Foreign  Lan¬ 
guage  Learning 

Mediunfi 

Repre 

Coding 

sentation 

Organisation 

Modality 

(Effector) 

Writing 

Keyboard 

Text 

Keyboard  ar- 
ranaement 

Gesture 

Selecting 

Mouse 

Text 

Hypertext 

Gesture 

Speaking 

Loudspeaker 

Speech 

Record  and  play¬ 
back  control 

Voice 

Table  6,  Example  for  the  application  of  the  multimedia  taxonomy  for  human  informa¬ 
tion  output 

Relations  to  other  taxonomies 

Based  on  this  terminology  the  area  of  representation  of  information  both  on  the  input  and 
the  output  side  of  the  human  user  must  be  further  differentiated.  Several  authors  propose 
classification  schemes  for  this  level  of  the  proposed  multimedia  taxonomy  (Table  7). 


Author(s) 

Classification  Schemes  for  Representation  (Coding  and 
Organisation) 

Arens  et  al,  1 993 

Temporal  endurance,  granularity,  baggage,  detectability, 
and  media  type 

Bernsen  1993 

Linguistic/nonlinguistic,  analog/nonanalog, 
arbitrary/nonarbitrarv,  and  static/dynamic 

Heller,  Martin  1995 

Elaboration,  representation,  and  abstraction 

Table  7.  Classification  Schemes  for  Representation  (Coding  and  Organisation). 


Five  categories  are  introduced  by  Arens  et  al.  1993:  temporal  endurance,  granularity, 
baggage,  and  detectability.  The  first  two  describe  time  and  resolution  aspects,  namely 
permanent  or  transient  and  continuous  or  discrete  properties.  Baggage  is  a  measure  of  the 
user’s  interpretation  effort,  and  detectability  characterises  the  intmsiveness  of  a 
presentation. 

Bemsen  1993  suggests  a  taxonomy  of  output  modalities  which  should  be  called 
representation  modes  according  to  the  taxonomy  proposed  in  this  paper.  He  applies  the 
categories  linguistic/nonlinguistic,  analog/nonanalog,  arbitraiy/nonarbitrary,  and 
static/dynamic. 

Heller  and  Martin  1995  describe  a  taxonomy  which  further  differentiates  the  represen¬ 
tation  level  called  media  expression  (Table  8).  They  allocate  three  types  of  expression 
namely  elaboration,  representation  and  abstraction  to  their  four  types  of  media  text. 
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graphics,  sound,  and  motion.  Elaboration  is  the  unchanged  original  version  of  the 
information,  e.  g.  the  medium  text  may  have  the  elaborations  free  text,  sentences  or  para¬ 
graphs.  Abbreviated,  stylised  modes  are  called  representation  (formatted  text,  drawings, 
etc.).  Finally,  abstraction  as  the  third  type  of  expression  is  based  on  metaphor  and 
standardised  forms.  In  summary,  media  expression  denotes  degree  of  abstraction,  author 
control,  and  design  effort  at  three  levels. 


Media  Type 

Elaboration 

Media  Expression 
Representation 

Abstraction 

Text 

Free  text,  sentences, 

paragraphs 

Bold,  italics,  head¬ 
lines 

Shapes,  icons 

Graphics 

Photographs,  rende¬ 
rings,  scanned  images 

Blueprints,  schema¬ 
tics 

Icons 

Sound 

Speech,  audio 
transcripts 

Intensity,  tone,  inflec¬ 
tion 

Sound  effects 

Motion 

Raw  film  footage 

Animation,  time- 
lapsed  photography 

Animated  models, 
highly  edited  video 

Table  8.  Media  expression  (Heller  and  Martin  1995), 


Conclusions 

It  has  been  shown  that  the  proposed  taxonomy  allows  a  more  systematic  description  of 
multimedia  human-machine  interation.  However,  instead  of  the  extensions  described  this 
taxonomy  remains  incomplete  as  the  area  of  humian  information  processing  was  widely 
excluded.  Obviously,  representation  and  modality  influence  the  ways  how  users  process 
perceived  information  and  how  they  generate  output  information.  Therefore  a  main  task 
in  the  future  development  of  multimedia  will  not  only  be  to  consider  the  properties  of 
human  sensor  and  effector  channels  but  also  to  support  cognitive  information  processing 
by  a  suitable  choice  of  medium,  representation  of  information,  and  modality. 
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Abstract 

This  paper  presents  a  method  for  modeling  and  comparing  different  multi-media  con¬ 
cepts.  The  focus  of  our  method  lies  on  a  social-oriented  evaluation  of  these  concepts.  We 
identified  and  systematized  elementary  characteristics  of  multi-media  systems  and  de¬ 
termined  a  set  of  frequently  used  building  blocks  to  support  the  construction  of  models 
representing  concepts  of  multi-media  applications.  The  models  can  be  used  to  visualize 
aspects  of  the  working  environment  and  to  support  early  discussion  between  different 
interest  groups.  To  support  the  evaluation  of  the  different  models  we  developed  a  proce¬ 
dure  consisting  of  several  if-then-rules  which  allows  to  find  out  the  weakpoints  of  multi- 
media  concepts  referring  to  various  social  aspects. 

Introduction 

Expecting  an  enormous  variety  of  multi-media  applications,  the  question  arises  of  how  to 
compare  different  multi-media  concepts  and  to  select  the  most  promising  approaches. 
Multi-media  comprises  every  kind  of  system  which  is  used  to  generate,  manipulate,  dis¬ 


tribute  and  receive  documents  or  transient  data  which  include  pictures  (fixed  as  well  as 
animated),  graphics,  sound,  speech,  text,  applets,  etc.  Furthermore  these  systems  can  be 
used  to  support  cooperation  and  communication;  therefore  groupware  and  multi-media 
will  be  increasingly  integrated.  Most  important,  these  systems  have  to  be  embedded  into 
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an  appropriate  organizational  structure  as  this  is  the  decisive  factor  for  success.  It  in¬ 
cludes  mostly  formal  as  well  as  informal  social  aspects.  To  anticipate  the  acceptance  of 
multi-media  applications,  besides  costs,  their  usability  and  usefulness  related  to  human 
needs  have  to  be  considered.  In  this  paper  we  present  a  modeling  and  evaluation  method 
that  is  oriented  towards  those  socio-technical  aspects. 

Stereotypes  and  socio-oriented  components  for  mulU-medm  applications 
To  identify  and  systematize  elementary  characteristics  of  multi-media  systems  we  ana¬ 
lyzed  concepts  in  the  hterature  as  well  as  prototypes  and  existing  apphcations.  Unlike 
classifications  which  are  based  on  the  available  types  and  classes  of  media  and  docu¬ 
ments  we  concentrated  on  blocks  of  typical  functionality  concerning  social  interaction. 
We  call  those  abstract  groupings  of  functionality  “multi-media  stereotypes”.  The  under¬ 
lying  idea  is  to  use  these  stereotypes  in  modeling  various  kinds  of  multi-media  systems. 
The  six  multi-media  stereotypes  in  figure  1  are  therefore  parts  of  our  modeling  notation. 

Through  the  refinement  and  synthesis  of  the  inspected  applications  we  determined  a  set 
of  frequently  found  precise  complexes  of  functionality.  We  call  them  building  blocks. 
These  “blocks”  can  repeatedly  be  used  to  “build”  different  multi-media  applications. 
With  this  method,  the  set  of  building  blocks  can  be  enlarged  if  necessary.  For  example,  a 
multi-media  Group  Decision  Support  System  is  “built”  by  “blocks”  such  as  moderation, 
evaluation,  whiteboard,  email  and  merging  of  data.  A  subset  of  elements  with  social 
relevance  (ref.  fig,  2)  was  identified  to  supply  a  basis  for  the  proposed  method. 

As  an  example  the  building  block  Authentication  is  described  in  detail.  Authentication  is 
related  to  the  social  aspect  authenticity  of  information.  By  authentication  one  can  verify 
the  assumed  origin  of  a  message  or  a  document.  Authentication  is  based  on  crypto  tech¬ 
nology.  In  principle  the  sender  (res.  author)  adds  an  authentication  key  to  the  information 
which  can  be  verified  by  the  addressee  using  the  appropriate  test  key.  The  process  of 


Anonymisation 

Data-Protection 

Parking 

Negotiation 

Data-Transfer 

Moderation 

Authentication 

Deanonymisation 

Checking-Rights 

automatic  Reply 

Document-Merging 

Giving-Rights 

AV -Communicati  on 

Safe-Payment 

Replication 

AV -Establishment-Phase 

Cipher-Decipher 

Tracing 

Valuation 

Filter 

Detour 

Call-Back 

Group-Lens 

Undo/Redo 

Data-Request 

History 

Whiteboard 

Data-Supply 

Identification 

Time-Cost-Control 

Fig.  2:  Social  relevant  building  blocks 

authentication  consists  of  the  following  steps  which  can  be  represented  by  blocks  of  our 
modeling  method:  exchanging  the  test  key  on  a  safe  communication  channel  where  its 
authenticity  can  be  guaranteed  by  a  trustworthy  third  party,  adding  an  authentication  key 
to  the  information  to  be  transmitted  and  checking  the  authenticity  after  transmission.  The 
authentication  checking  service  is  started  by  entering  the  test  key  and  the  pair  of  authen¬ 
tication  key  and  information.  As  the  result  it  reports  the  reliability  of  the  inspected  in- 
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formation.  It  is  sensible  to  describe  the  complex  structure  of  authentication  by  a  model. 
One  can  imagine  different  concepts  being  possible  to  organize  authentication  and  to  em¬ 
bed  it  into  multi-media  applications.  These  different  concepts  can  also  be  modeled. 

Modeling  multi-media  systems  for  the  socio-oriented  evaluation 
The  usage  of  models  is  especially  intended  to  compare  different  concepts  of  multi-media 
applications.  The  comparison  is  intended  to  be  socio-oriented  which  means,  that  aspects 
of  the  working  environment  of  users  are  visualized  with  diagrams  serving  as  a  foundation 
of  socio-oriented  evaluation.  The  method  for  the  socio-oriented  evaluation  of  models  is 
described  below. 

It  must  be  pointed  out  that  modeling  can  be  constructive  and  re-constructive.  For  the 
purpose  of  discussing  different  concepts  for  the  realization  of  socio-technical  systems,  a 
modeling  method  must  be  especially  designed  for  the  construction  of  such  a  system.  The 
representation  of  social  aspects  and  requirements  is  helpful  to  support  an  early  discussion 
between  different  interest  groups.  Therefore  we  have  identified  the  presented  set  of  socio- 
oriented  building  blocks.  It  is  advantageous  that  the  notation  and  semantics  of  the  mod¬ 
els  are  based  on  standard  software  engineering  or  business  process  reengineering  models 
to  avoid  multiple  recording  of  the  same  facts:  existing  models  can  be  used  for  the  socio- 
oriented  evaluation  and  socio-oriented  models  should  be  usable  to  implement  a  system. 

In  the  following  paragraphs  we  describe  which  requirements  for  a  socio-oriented  model¬ 
ing  notation  exist  and  how  to  meet  them  with  an  object-oriented  notation.  Object- 
oriented  technology  is  especially  suitable  for  component-based  systems  like  multi-media 
systems. 

The  representations  created  by  modeling  methods  should  comply  with  ergonomic  crite¬ 
ria,  This  is  especially  relevant  if  the  models  are  used  for  a  participatory  evaluation  of  the 
modeled  concepts.  Easiness  of  creation,  comprehensibility,  communicability  for  model¬ 
ers  and  for  recipients  must  be  as  good  as  possible  to  support  the  discussion  of  different 
realization  concepts  for  the  multi-media  system. 

One  approach  to  develop  a  modeling  method  for  our  purposes  is  to  extent  existing  meth¬ 
ods,  However,  it  might  prove  that  this  extension  leads  to  a  new  method.  In  this  paper  we 
start  with  a  modeling  notation  which  is  based  on  the  object-oriented  methodology  intro¬ 
duced  by  Ivar  Jacobson  [Jacobson  et  al.  1992]  and  on  the  extensions  for  process  model¬ 
ing  [Jacobson  et  al.  1995].  We  adapted  and  extended  the  notation  by  adding  stereotypes 
analogous  to  Jacobson’s  extensions  for  business  process  modeling  and  using  modeling 
concepts  from  UML  [Booch  et  al.  1997].  UML  is  a  rising  standard  for  object-oriented 
modeling  and  there  already  exist  a  mapping  from  the  notations  of  Jacobson  to  the  UML. 
With  this  mappings  it  is  possible  to  transform  models  from  the  usage  oriented  perspec¬ 
tive  to  an  implementation-oriented  perspective. 

Basically,  in  a  socio-oriented  modeling  notation,  objects,  activities,  roles  and  relations 
must  be  representable  [Herrmann  et  al,  1997]. 

Objects  means  physical  objects  as  well  as  virtual  objects  in  an  object-oriented  sense.  As 
multi-media  systems  are  built  from  components  as  mentioned  above,  object-oriented 
modeling  techniques  fit  properly  because  of  the  independence  of  objects  representing 
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these  components. 


Activities  represent  the  actions  of  users  and  participants  or  system  based  processes  in  the 
socio-oriented  system.  Therefore  it  is  necessary  for  the  modeling  notation  to  be  able  to 
visualize  the  dynamic  aspects  of  the  system  to  catch  the  sequences  of  actions  . 

Roles  are  a  concept  to  abstract  from  real  world  participants.  This  is  very  helpful  in  mod- 
ehng  socio- technical  systems. 

Relations  between  these  concepts  must  be  representable  to  fix  all  the  dependencies  be¬ 
tween  objects,  activities  and  roles.  An  example  of  relations  is  the  usage  relation  between 
different  objects  or  objects  and  activities. 

The  selected  modeling  methodology  from  Jacobson  et  al.  (OOSE)  includes  different 
types  of  models.  In  Use  Case  Models  it  is  possible  to  visualize  roles,  activities  and  rela¬ 
tions  between  them.  The  object-model  focuses  on  objects  (types)  and  their  relations. 

As  one  extension  to  the  modeling  notation  we  use  the  presented  stereotypes  to  visualize 
the  abstract  functionality  of  components  of  the  multi-media  application  (see  fig.  3). 

When  modeling  multi-media  systems  it  is  especially  necessary  to  visualize  the  flow  of 
information,  as  multi-media  systems  make  communication  channels  available.  This  flow 
of  information  is  fundamental  for  the  socio-technical  systems.  It  must  be  transparent 
which  pieces  of  information  flow  over  the  recognized  channels  and  where  the  source  and 
destination  of  this  information  lie.  For  this  requirement  we  propose,  in  addition  to  the 
OOSE-Notation,  the  concept  of  association  classes  from  UML,  which  makes  it  possible 
to  associate  types/classes  to  relations.  The  associations  in  the  object  models  are  therefore 
limited  to  unidirectional  relations  to  prevent  the  hiding  of  the  flow  of  information. 

It  is  helpful  to  start  the  modeling  process  with  modeling  the  business  with  the  Use  Case 
Method  to  figure  out  the  functional  requirements  for  the  multi-media  system.  The  design 
of  the  application  with  predefined  building  blocks  can  be  based  on  such  a  model.  With 
regard  to  the  dynamic  development  in  the  field  of  multi-media  systems,  the  presented 
building  set  is  expandable  by  adding  or  modifying  building  blocks  and  evaluation  rules. 
For  the  social  evaluation  it  is  even  necessary  to  catch  those  elements  with  social  rele¬ 
vance.  We  introduced  the  socio-oriented  components  to  support  that.  The  modeling  pro¬ 
cess  is  driven  by  those  elements.  Models  will  be  refined  until  these  elements  are  recog¬ 
nized  or  until  it  is  obvious  that  they  are  necessary  or  it  can  be  stated  that  these  elements 
are  not  necessary  anyway. 

It  is  also  necessary  to  note  different  socio-oriented  attributes  to  the  types  in  the  model. 
They  are  used  for  the  socio-oriented  evaluation  described  in  the  following  section  and  for 
testing  the  completeness  of  the  model.  Therefore  we  use  rules  also  presented  in  the  fol¬ 
lowing  section.  The  rules  for  modeling  include  questions  and  additions  to  the  current 
model  which  make  the  social  necessity  of  functionality  visible  (see  fig.  7). 

As  an  example  we  describe  parts  of  the  model  for  an  insurance  company.  In  the  insur¬ 
ance  organization  new  contracts  from  sales  representatives  are  first  checked  by  office 
workers.  After  the  first  checking  and  correction  a  new  workflow  process  should  be  initi- 
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ated.  The  communication  and  the  transport  of  new  contracts  should  be  supported  with  a 
multi-media  environment.  This  business  system  can  be  modeled  with  Use  Cases. 


Fig.  3:  Insurance  Office  Example  -  Workplace  Object  System 


The  object  model  of  one  possible  concept  for  the  multi-media  application  is  shown  in  fig. 
3.  It  represents  the  multi-media  system  from  the  perspective  of  the  office  worker. 

First  it  is  necessary  to  describe  the  used  elements  from  Jacobson.  The  notation  consists  of 
types  visualized  by  a  circle.  There  are  three  basic  stereotypes  for  types:  control  types 
have  controlling  functionality  on  the  object  system,  visualized  by  an  arrowhead,  bound¬ 
ary  types  represent  interfaces,  visualized  by  a  „T‘-adomment  and  entity  types  have  in¬ 
formation  storage  functionality,  visualized  by  a  baseline.  The  modeling  language  for 
business  processes  consists  of  different  other  stereotypes.  Especially  for  participants  in 
the  business  system  Jacobson  used  special  symbols  like  the  case  worker  stereotype  (Of- 
fice  Worker).  Arrows  represent  relations  in  the  object-oriented  sense. 

The  object  model  contains  several  examples  for  the  usage  of  building  blocks  and  multi- 
media  stereotypes.  The  AV-Communication  building  block  is  used  for  the  communica¬ 
tion  between  the  office  worker  and  the  sales  representative,  visualized  by  the  synchro- 
nous-communication- stereotype.  Another  communication  channel  called  File  Transfer 
System  is  used  for  the  transmission  of  files,  especially  for  the  application  form.  Trans¬ 
mitted  forms  are  stored  in  a  document  image  system.  The  document  image  system  has 
database  functionality,  represented  by  the  database-stereotype.  If  necessary  the  more  spe¬ 
cialized  stereotypes  central  database  or  distributed  database  could  be  used.  For  further 
processing  in  the  organization  a  workflow  management  system,  that  is  based  on  the 
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document  image  system  for  the  storage  of  forms,  is  used.  The  basic  functionality  of  a 
workflow  management  system,  the  coordination  of  the  cooperation,  suggests  using  the 
coordinated  cooperation  stereotype. 

As  an  example  for  adding  attributes  to  types,  the  attribute  confidentiality  is  noted  to  the 
application  form.  Attributes  are  usually  invisible  in  the  graphical  model.  To  show  the 
usage  of  rules  the  following  example  may  be  impressive.  We  specified  the  following 
modeling  rule  to  ensure  the  social  aspect  authenticity  of  information: 

If  the  document’s  confidentiality  is  high  AND  its  violation  of  identity  has  to  be  considered 
then  check  the  model  for  existence  of  building  block  “authenticity  checking”.  _ 


Fig.  4:  Insurance  Office  Example  -  Document  Imaging  System  with  Authentication 


The  more  detailed  Object  System  DocImageSystem  with  Authentication  (see  fig.  4)  con¬ 
sists  of  a  user  interface  for  the  office  worker  (Office  Worker  UI)  and  the  underlying 
document  image  system  (DocImageSystem)  that  realizes  the  persistence  of  the  incoming 
forms.  As  well  we  can  identify  the  required  building  block  Authenticity  Checking  Serv~ 
ice.  Its  functionality  has  been  described  above. 

Comparative  Evaluation 

We  derive  a  set  of  if-then-rules  referring  to  the  literature  in  the  field  of  ergonomics  and 
technology  assessment  [e.g.  Henmann,  94;  VDI,  91].  These  rules  exploit  the  social  as¬ 
pects  to  make  comparative  evaluations  between  different  multi-media  application  con¬ 
cepts  and  to  find  out  which  of  them  are  more  advantageous  if  various  social  aspects  are 
taken  into  account.  In  addition  to  building  blocks  (see  fig.  2),  we  identified  structures 
(see  fig.  5)  with  social  relevance  by  referring  to  the  experience  we  gained  by  modeling 
multi-media  applications.  By  the  term  structures  we  refer  to  phenomena  which  cannot  be 
represented  by  single  building  blocks  because  they  are  distributed  over  several  blocks  or 
expressed  by  the  relationships  between  them. 

Examples  for  socially  relevant  structures: 

•  Choice  of  communication  partner _ _ 
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•  Choice  of  communication  means 

•  Attainability  of  roles 

•  Transparency  to  the  participants _ _ _ _ _ 

Fig.  5:  Examples  for  socially  relevant  structures 

To  each  socio-oriented  building  block  we  added  attributes  (see  fig.  7)  which  are  relevant 


Examples  for  attributes: 

•  personal  data  included:  (yes,  no) 

•  sensibility:  (low,  average,  high) 

these  three  values  mark  the  intensity  of  possible  negative  consequences  for  individual 
and  for  business  concerns  which  can  be  the  result  of  the  publication  of  data 

•  confidentiality:  (low,  average,  high) 

•  universal:  (yes,  no) 

This  attribute  relates  to  the  availability  of  data  for  all  participants.  _ 

for  the  socio-oriented  evaluation  and  which  are  also  used  in  the  if-then-rules. 

Fig.  6:  Additional  attributes  for  social  evaluation 

Our  method  allows  a  -  mostly  comparative  -  evaluation  of  multi-media  applications  re¬ 
garding  different  social  values,  such  as: 

•  data  protection 

•  security 

•  privacy 

•  authenticity  of  information 

•  chances  for  participation 

•  reduction  of  the  context 

•  transparency 

There  are  too  many  rules  to  completely  present  them  in  this  article.  To  give  an  example, 
figure  7  shows  a  subset  of  the  if-then-rules  to  evaluate  multi-media-applications  in  de¬ 
pendency  of  “data-protection”. 
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Examples  of  if-then-rules  for  the  social  aspect  data-protection: 

2.  if  “personal  =  yes”  has  been  noted  at  the  building  block 

then  check  the  existence  of  following  building  blocks  in  the  model: 

•  making  data  anonymous 

•  coding 

•  negotiation 

•  tracing 

•  identification 

4,  if  “sensibility  =  high”  has  been  noted  at  the  building  block 

then  check  the  existence  of  following  building  blocks  in  the  model 

•  checking  the  rights 

•  coding 

•  tracing 

9.  if  the  building  block  “Control  of  costs  and  time”  is  part  of  the  model 
then  check  the  existence  of  following  building  blocks  in  the  model: 

•  making  data  anonymous 

•  checking  the  rights 

Fig.  7:  Examples  of  evaluation  rules  for  the  aspects  "data  protection" 
Evaluation-procedure 

To  evaluate  a  multi-media  concept  we  suggest  constructing  a  list  of  weakpoints  for  the 
multi-media  concepts  being  regarded.  This  list  allows  a  comparison  of  differently  con¬ 
structed  concepts  on  the  basis  of  measurable  values.  The  list  of  weakpoints  can  be  used 
to  serve  three  different  purposes:  it  gives  hints  to  improve  a  concept  of  a  multi-media 
application,  it  might  contain  weakpoints  showing  that  the  concept  is  clearly  violating 
certain  social  norms,  it  might  become  obvious  (in  relationship  to  certain  aspects)  that 
one  concept  is  more  advantageous  than  another  one. 

To  construct  a  list  of  weakpoints,  we  suggest  using  the  following  procedure  : 

(1)  Choose  the  aspect  for  the  evaluation  (e.g.  data  protection). 

(2)  Start  with  the  first  evaluation  rule  for  the  chosen  aspect. 

(3)  For  each  object  and  its  relationship  to  others  investigate  every  rule:  if  it  is  violated,  a  hint  to  this 
weakpoint  is  added  to  the  list. 

(4)  For  any  further  rules  proceed  the  same  way  as  described  in  step  (3) _ 

With  this  procedure  we  can  construct  a  values  list  for  every  social  aspect  ,  which  we 
chose  to  be  part  of  the  evaluation  or  comparison  of  the  regarded  multi-media  concepts. 

The  evaluation  rules  and  the  procedure  are  independent  of  the  used  modeling  language,  if 
the  modeling  language  supports  the  concept  of  building  blocks  and  refinement. 

Conclusions 

We  presented  expandable  methods  for  the  socio-oriented  comparing  of  multi-media  ap¬ 
plication  concepts.  Therefore,  we  introduced  building  blocks  to  support  modeling  of 
these  multi-media  concepts  and  to  indicate  the  socio-oriented  aspects.  The  relevance  of 
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the  proposed  building  set  has  been  demonstrated  in  this  paper  by  an  example. 

The  modeling  notation  is  based  on  standard  methodologies  and  is  independent  from  the 
building  set  concept.  The  shown  object-oriented  modeling  language  is  appropriate  for 
component-based  systems  like  multi-media  systems.  However,  additional  extensions  are 
still  needed  to  make  social  aspects  more  comprehensible.  This  is  especially  necessary  if 
the  social  aspects  are  related  to  relationships  between  objects  or  to  the  overall  structure  of 
parts  of  the  model. 

Also,  the  evaluation  rules  and  the  methods  for  the  evaluation  are  independent  from  the 
used  modeling  notation.  The  method  for  the  evaluation  leads  to  a  list  of  ratings  support¬ 
ing  a  comparison  of  multi-media  concepts  and  to  hints  for  improving  them. 
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Abstract 

Virtual  Environment  (VE)  describes  a  computer-mediated  experience  by  which  an 
operator  perceives  a  simulated  environment  by  means  of  special  human-computer  inter¬ 
face  equipment.  The  human  operator  interacts  with  virtual  objects  in  the  environment  as 
if  they  were  real.  Several  operators  are  able  to  interact  and  co-operate  in  a  shared  virtual 
environment,  such  as  a  synthetic  Tactical  Situation  Display. 

The  Research  Institute  for  Electronics  and  Mathematics  (FFM)  carries  out  a  research  pro¬ 
ject  which  has  the  goal  of  acquiring  methods  for  thi*ee  dimensional  visualisation  of  tacti¬ 
cal  situation  data  and  interaction  between  the  officer  in  command  and  the  units 
visualised  on  the  display  system. 

For  fulfilling  the  operational  task  several  operator  look  at  a  three  dimensional  representa¬ 
tion  of  a  landscape  using  VE-devices  like  shutter  glasses,  etc.  .  The  ergonomic  research 
work  is  to  design  and  evaluate  the  interactive  functions.  The  important  subject  is  the  in¬ 
vestigation  of  different  control  input  devices  for  simultaneous  interaction  of  operators  at 
a  common  virtual  Tactical  Situation  Display  unit  (TSD),  An  interaction  task  is  e.g.  to 
move  and  position  graphic  objects  as  well  as  tactical  symbols. 

The  first  step  to  be  done  is  the  development  of  principles  for  the  design  of  a  human-ma¬ 
chine-interface  which  enables  the  user  to  handle  tactical  situation  data  in  a  semi-immer¬ 
sive  way  and  to  interact  with  his  own  units. 

The  long-term  purpose  of  the  realised  experimental  facility  of  an  Electronic  Sandtable 
(ELSA)  is  to  perform  research  work  with  respect  to  development,  performance,  evalua¬ 
tion,  and  optimisation  of  compliant  terrain  database,  real-time  communications,  and  net¬ 
working  of  heterogeneous  "Command  &  ControF-systems  computing  graphic  situation 
display  information. 

This  paper  describes  technical  challenges  and  discusses  the  issue  of  feasibility  of  a  pro¬ 
ject  started  at  the  Research  Institute  for  Electronics  and  Mathematics  (FFM). 


Introduction 

Virtual  Environment  (VE),  also  popular  under  the  synonyms  Virtual  Reality  (VR)  or 
Cyberspace,  describes  a  computer-  mediated  experience  by  which  an  operator  perceives  a 
simulated  environment  by  means  of  special  human-computer  interface  equipment.  The 
human  operator  interacts  with  virtual  objects  of  the  environment  as  if  they  were  real.  VE 
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is  conditioned  by  two  factors,  which  are  immersion  and  interaction  (du  Pont,  1994).  The 
effect  of  immersion,  the  subjective  "being  there"-feeling  of  the  user  is  increased  by  most 
complete  stimulation  of  the  human  senses.  Interaction  is  realised  by  a  variety  of 
interaction  techniques  which  guarantee  an  intuitive  behaviour  of  the  operator  in  case  of 
interacting  with  virtual  objects. 


VE  has  become  popular  during  the  last  years  due  to  the  rapid  development  and  enhance¬ 
ment  of  computers,  especially  in  the  field  of  workstations  with  high  graphic  power. 
Powerful  tools  for  generating  synthetic  environments  have  been  developed  and  sold,  too, 
and  it  is  only  a  question  of  time,  until  new  approaches  for  the  design  of  human-machine- 
interfaces  (HMI)  will  have  grown  up. 

Promising  ideas  of  an  immersive  interaction  between  human  beings  and  technical 
systems  have  been  created  for  the  civilian  world,  e.g.  for  medical  purposes,  architecture, 
education,  systems  design,  etc.,  and  for  the  military  world.  But  it  is  well  known  from  the 
history  of  technical  development,  that  the  promises  of  new  approaches  and  ideas  are 
often  very  high  and  easily  made.  But  after  critically  studying  the  techniques  differences 
between  promises  on  the  one  hand  and  technique’s  benefits  and  risks  on  the  other 
appear. 

Especially  for  applications  connected  with  high  risks  for  the  operator  and  the 
environment  the  consequences  of  using  these  new  techniques  have  to  be  examined  very 
closely.  Since  the  consequences  of  failures  in  using  or  handling  such  systems  are  risky  or 
perilous  it  has  to  be  ensured  that  the  interaction  techniques  support  reliable  and  safe 
function  of  the  systems.  An  important  influence  on  the  handling-safety  is  given  by  the 
early  involvement  of  ergonomics  in  the  system  design  process.  By  determination  of  the 
human  machine  interface’s  requirements  it  is  be  guaranteed  that  the  system  fits  the 
operator  which  is  an  important  precondition  for  systems  design  and  handling-safety. 
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The  current  state  of  military  commander*s  workplaces 

Before  projecting  the  idea  of  a  virtual  Tactical  Situation  Display  (TSD)  it  should  be  ex¬ 
plained  what  kind  of  functions  are  performed  by  means  of  such  a  device.  TSDs  are  lo¬ 
cated  in  operation  centres  of  command  posts.  They  are  used  to  display  the  current  situa¬ 
tion  of  own  and  reconnoitred  enemy  troops  and  facilities,  e.g.  positions  of  units,  head¬ 
quarters,  bridges,  mine  barriers,  etc,  in  the  operation  area  to  the  commander  of  a  unit, 
e.g.  a  commanding  officer  of  a  tank  brigade. 

The  quantity  of  displayed  situation  information  depends  upon  the  availability  of  com¬ 
mand  and  control.  Especially  communication  systems  like  radio  telephony  which  are 
used  to  receive  and  send  situation  data  at  present  have  high  importance.  Due  to  the 
danger  of  becoming  reconnoitred  by  the  enemy’s  communications  intelligence  there  are 
restrictions  on  the  use  of  communication  systems.  Furthermore  communication  links  are 
attentionally  jammed  by  the  enemy  so  that  the  transmission  quality  is  limited. 

Moreover  the  quality  of  displayed  situation  data  depends  on  the  accuracy  of  the  own 
troop’s  reports  about  the  own  situation  and  the  observations  made  by  the  soldiers  or  by 
technical  reconnaissance  systems. 

Moreover  the  TSD  is  used  for  tactical  planning  of  intended  future  operations.  Quantity 
and  quality  of  situation  data  are  essential  for  an  adequate  operation  planning. 

At  present  the  paper  &  pencil  technique  is  used  to  display  these  pieces  of  information. 
Situation  information  data  are  manually  noted  on  the  Tactical  Situation  Display.  There  is 
a  substantial  set  of  symbology  and  colours  used  for  the  different  kinds  of  information, 
e.g.  for  distinction  between  own  and  enemy  troops,  for  kinds  of  vehicles  (battle  tank, 
infantry  fighting  vehicle,  etc.),  for  kinds  and  states  of  barriers  (prepared,  permanent, 
time-limited)  and  so  forth.  Due  to  this  technique  it  is  a  time-consuming  work  to  hold  this 
display  up  to  date  while  receiving  actual  information  from  other  units.  Furthermore  the 
transmission  of  orders  for  operations  by  means  of  radio  telephony  takes  a  lot  of  time. 
Due  to  these  facts  a  situation  of  high  information  flow  and  high  workload  of  the 
personnel  requires  the  whole  operator’s  capacity  and  might  even  lead  to  states  of 
overload.  Figure  2  gives  an  impression  of  the  situation  in  today’s  operation  centres. 

It  is  obvious  that  the  form  of 

•  displaying  situation  information  data  at  the  commander’s  workplace  by 
means  of  paper  &  pencil  TSD  and 

•  communication  mainly  made  by  radio  telephony 
have  not  significantly  changed  for  at  least  50  years. 
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Figure  2:  Today’s  army  control  centres 


Technical  developments  of  C*I-Systems 

The  technical  environment  has  developed  in  recent  years  and  will  be  improved  in  future 
with  high  speed.  This  development  has  taken  and  will  take  place  in  several  fields  which 
are  usually  joined  within  the  expression  ^-(Command,  Control,  Communications, 
Computers  and  Intelligence)  systems: 

•  Command/Control:  Data  hnks  between  operation  centres  attached  to  networks 
will  improve  amount  and  availability  of  situation  information.  Furthermore 
these  means  will  reduce  the  necessity  to  use  radio  telephony  for 
communication. 

•  Communication:  Radio  telephony  by  means  of  digital  transmission  reduces 
the  vulnerability  of  jamming  and  spying  by  the  enemy. 

•  Computer:  The  use  of  computers  for  military  applications  will  rise  due  to  low 
costs  and  necessity  to  process  huge  amounts  of  data. 

•  Intelligence:  New  sensors,  e.g.  FLIR  (Forward  Looking  InlraRed),  improve 
the  quality  and  quantity  of  observations. 

These  developments  will  influence  two  factors  of  the  workplace  at  an  operation  centre: 

•  increasing  amount  of  available  data, 

•  non-verbal  data  reception  via  computer  networks. 

Considering  the  information  flow  caused  by  technical  developments  it  can  be  estimated 
that  current  techniques  for  handling  and  interpretation  of  tactical  situation  data  will  be¬ 
come  insufficient  within  the  next  future. 

For  this  reason  computer-based  command  and  control  information  systems  (CCIS)  have 
been  developed  for  the  services  (army,  navy,  and  airforce).  In  the  following  CCIS 
developments  for  the  German  army  will  be  discussed.  The  arguments  are  likely  to  be 
valid  for  the  other  services  as  well. 
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Examples  for  CCIS-developments  within  the  German  army  are  the  CCISs  HER  OS 
(Heeres-Fuhrungsinformationssystem  fur  rechnergestUtzte  Operationsflihrung  in  Staben) 
for  the  army  command,  and  ADLER  (Artillerie  Daten-,  Lage-  und  Einsatz- 
Rechnerverbund)  for  the  artillery.  The  current  state  of  CCIS  development  has  its  focus  on 
data  transmission  between  units.  It  is  planned  and  partly  realised  to  connect  the  army’s 
units  via  networks  so  situation  information  data  and  orders  can  be  exchanged  in  non¬ 
verbal  manner.  In  order  to  make  these  systems  "open"  for  communication  with  other 
services  of  the  German  Armed  Forces,  NATO  and  WEU  partner  nations  protocols  and 
interfaces  have  been  standardised.  Figure  3  illustrates  the  structure  of  such  networks. 


Figure  3:  CCIS-scenario  (reprint  with  permission  of  SIEMENS  AG  -  Sicherungstechnik) 


The  hardware  of  the  CCISs  human-machine-interface  has  mostly  been  realized  in  a  con¬ 
ventional  way  like  civilian  computer  workplaces  using  commercial  of  the  shelf  (COTS) 
products,  i.e.  computer  displays,  keyboards,  and  additional  input  devices  like  mouse  or 
trackball. 

Present  CCISs  visualise  the  situation  information  data  in  a  two-dimensional  way.  The 
symbology  is  also  similar  to  the  one  used  with  the  paper  &  pencil  TSDs.  Tools  like  a 
march-planner  or  the  display  of  lines  of  sight  allow  the  user  to  take  into  account  terrain 
shape,  cruising  speed,  etc.,  when  planning  a  march  or  checking  whether  an  object  is 
visible  to  another  one.  The  march-planner  is  important  in  rough  terrain,  display  of  lines 
of  sight  in  order  to  determine  acceptable  positions  of  own  troops  for  intended  operations. 
Current  CCIS  techniques  bring  along  a  significant  improvement  of  the  situation  at  the 
commanders’s  workplace. 

Nevertheless  it  is  obvious  that  the  development  will  go  on  in  future.  This  shall  be 
explained  by  two  examples:  One  disadvantage  of  two-dimensional  representations  of 
three-dimensional  data  is  the  necessarity  of  creating  a  mental  image  of  the  terrain. 
Although  assisting  tools  are  implemented  it  can  be  assumed  that  the  availability  of  three- 
dimensional  Geo-Information- Systems  (GIS)  with  terrain  databases  like  DTED  or  DHM, 
and  feature  vector-databases  like  DFAD,  DCW,  VMAP  and  others  means  improvement. 
Other  improvements  can  be  made  by  a  perspective  view  from  any  location  of  the 
scenario  into  any  direction.  The  operator  would  be  able  to  see  the  terrain  and  objects  in 
the  same  way  a  tank  commander  would  do,  which  might  become  an  important  function 
in  some  situations. 
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An  important  ergonomic  requirement  of  human-machine-interface  (HMI)  design  is  to 
allow  an  intuitive  interaction  between  the  operator  and  the  objects  on  the  display.  In  pre¬ 
sent  CCIS  the  interaction  with  objects  of  the  scenario  is  not  included  in  a  satisfactory 
way.  Instead  the  operator  has  to  follow  various  procedures  to  manipulate  objects. 
Although  most  of  the  systems  are  menu-driven,  the  procedures  take  an  important  amount 
of  time  and  training  for  learning  to  use  the  systems. 

These  two  circumstances  are  only  examples  where  lacks  of  the  present  systems  appear 
and  improvements  can  be  achieved  by  applying  new  display  and  interaction  techniques. 

Existing  visions  of  future  C"*!  -Systems  describe  a  combination  of  new  sensors, 
communication  lines  and  display  techniques.  The  Swedish  ’’Mobile  Joint  Command  and 
Control  System  2010"  (ROLF),  a  vision  of  a  future  mobile  Command  &  Control  - 
System,  will  use  a  device  called  AQUARIUM  as  TSD.  The  AQUARIUM  consists  of  a 
large  semi-inunersive  workbench  (which  will  be  described  later)  to  enable  discussion  and 
co-operation  between  different  operators.  The  underlying  concept  of  ROLF  is  variable.  It 
is  principally  military,  but  it  can  be  used  for  civil  applications,  e.g.  supporting  the  society 
under  severe  strain  and  stress  in  case  of  catastophy,  too.  (Sundin,  1996) 

The  US -American  system  of  the  Naval  Research  Laboratory  (NRL)  or  the  system 
MIRAGE  of  the  Institute  for  Simulation  and  Training  (1ST)  use  exclusively  workbenches 
as  TSDs  (NRL,  1997;  1ST,  1997).  The  systems  were  already  tested  in  military  exercises 
and  found  very  useful  and  advantageous  compared  to  conventional  TSDs.  Especially  the 
possibility  of  displaying  "natural  features"  of  three-dimensional  terrain  in  combination 
with  abstract  information  data  (visibility,  range  of  weapon  systems,  etc.)  was  found  to  be 
very  useful  (Wilson,  1997).  Another  important  result  was  that  there  is  special  need  for 
ergonomic  user  interfaces  and  training  to  use  the  new  technology  and  its  benefits. 

Virtual  Environment  Systems  -  A  brief  overview 

In  recent  years  there  have  been  a  lot  of  approaches  and  ideas  for  future  displays  and 
interaction  techniques  in  the  field  of  Virtual  Environments.  The  devices  available  on  the 
market  are  manifold.  A  good  structure  of  VE-systems  is  given  by  use  of  the  degree  of 
immersion,  which  means  "to  what  extent  a  user  is  physically  tied  in  a  virtual 
environment"  (Bullinger,  1997). 

Simple  VE-systems  have  nearly  no  immersive  effect.  Such  systems  use  low-cost  desktop- 
computers  and  usual  monitor-displays.  The  virtual  environment  is  presented  only  in  two- 
dimensions.  Real  three-dimensional  navigation  or  interaction  is  very  limited.  An  example 
of  these  systems  are  commercial-of- the- shelf  software  (games,  etc.)  or  VRML-browser 
on  conventional  PCs. 

Advanced  (semi-immersive)  systems  create  a  higher  degree  of  immersion.  A  monitor  or 
projection-plane  serves  as  display  and  by  use  of  shutter  glasses  or  other  techniques  three 
dimensional  visualisation  is  achieved.  Interaction  devices  with  6  degrees-of-freedom 
(DOF)  are  often  used.  Examples  for  semi-immersive  systems  are  graphic  high-end 
workstations  with  monitors,  projectors  or  with  a  virtual  workbench.  AU  these  systems 
make  the  virtual  environment  and  objects  become  a  part  of  our  real  world  (Bullinger, 
1997;  Kruger  &  Frohlich,  1995). 
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High-end  (full-immersive)  systems  provoke  total  visual  immersion  of  the  user  into  the 
virtual  environment.  These  systems  create  a  virtual  world  and  let  the  human  operator 
participate  in  it.  Navigation  and  interaction  is  possible  in  three  dimensions  and  three 
directions.  Examples  for  such  systems  are  the  head  mounted  displays  (HMD),  special 
1/0-devices  or  the  CAVE  (Cruz-Neira  et  al.,  1993). 

Because  of  the  rapid  evolution  and  the  variety  of  VE-systems  and  devices  it  does  not 
seem  to  be  useful  to  put  in  a  summary  at  this  point.  To  get  an  impression  about  the  de¬ 
vices  available  at  the  market  it  is  recommended  to  refer  to  further  articles  (Bullinger, 
1997;  Kalawsky,  1993;  Grandt  &  Gartner,  1997). 

The  Idea  of  a  Tactical  Situation  Display  using  VE-Technology 
Decision  making  is  strongly  influenced  by  the  information  available  and  perceivable.  For 
this  reason  the  human  factor  is  the  most  important  part  to  define  the  needs  and  goals  of 
an  advanced  TSD, 

TSDs  should  include  a  variety  of  diverse  information,  which  starts  at  terrain  information 
and  position  of  units  and  ends  at  social  and  political  information  of  the  area  of  interest. 
The  information  available  should  be  as  complete  and  actual  as  possible.  These  goals  are 
technical  and  with  ongoing  improvement  of  sensors,  communications  and  information- 
technology  it  is  only  a  question  of  time  until  availability  of  information  is  sufficient. 

Because  of  the  visualisation  of  huge  amounts  of  information  data  the  use  of  virtual  en¬ 
vironment  display  techniques  has  been  discussed  and  found  very  promising.  The  real-life 
appearance  of  the  information  is  supposed  to  create  a  high  situation  awareness,  which 
will  make  decision  making  easier  for  the  human  operator.  Moreover  VE  has  been  found 
very  useful  for  visualisation  of  huge  amounts  of  complex  data  in  several  other  research 
studies. 

Additional  benefits  for  using  VE-technology  as  TSD  is  the  possibility  of  including  as¬ 
sisting  functions  for  preliminary  decision  making  and  assessing  the  effects  of  an  intended 
operation.  Such  functions  might  be  the  visualisation  of  the  range  of  weapon  systems, 
visualisation  of  the  range  of  units  for  transportation,  information  about  the  current 
airspace  situation  above  a  terrain,  lines  of  sight,  visibility  and  others.  Such  functions 
would  make  the  TSD  an  assisting  tool  for  tactical  situation  assessment. 

Moreover  co-operation  is  very  important  at  operation  centres.  A  battle  commander 
should  have  the  possibility  to  discuss  his  decisions  and  alternative  ideas  with  his  staff. 
Although  there  are  some  thoughts  about  "virtual  staff  centres",  yet  real  co-operation 
cannot  take  place  in  full- immersive  Virtual  Environments  due  to  unknown  co-operation 
techniques  and  social  problems.  Instead  the  virtual  world  should  become  a  part  of  the 
real  world  where  staff  meets  as  real  persons,  not  as  avatars  or  virtual  agents. 

For  this  reason  it  was  decided  to  use  a  virtual  workbench.  The  workbench  looks  like  a 
conventional  cardtable  and  by  using  shutter  glasses  it  is  possible  to  create  three- 
dimensional  images  on  its  horizontal  projection  plane.  This  way  virtual  objects  become  a 
part  of  the  real  world.  Especially  co-operation  and  interaction  with  the  graphic  objects  is 
more  natural  than  it  would  be  by  using  full-immersive  systems. 
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The  Electronic  Sandtable  (ELSA)  -  Facility  at  the  FFM 

The  Electronic  Sandtable  (ELSA)  at  the  Research  Institute  for  Electronics  and  Mathe¬ 
matics  (FFM)  has  been  designed  as  an  advanced  experimental  three-dimensional  display 
system  for  tactical  information.  It  is  planned  to  equip  the  system  with  additional 
functions  to  extend  the  passive  display  system  to  an  assisting  system  for  staff  members  in 
command  centres  (Alexander  et  al,  1997). 

To  get  a  comparison  to  conventional  TSDs  the  ELSA-project  has  been  designed 
according  to  the  following  demands: 

•  display  of  tactical  information  of  existing  CCIS, 

•  use  of  standardised  tactical  symbols  according  to  ZDV  1/11, 

•  use  of  military  geographic  data  (AMilGeo,  1996), 

•  reference  to  the  usual  scale  of  military  paper-cards. 

As  it  can  be  seen  in  figure  4  the  structure  of  the  display-system  can  be  divided  into  two 
main  parts. 

Part  1  works  offline  and  creates  the  static  scene-database  to  be  displayed.  Geographic 
data  (terrain,  feature  and  satellite  data)  and  detailed  objects  (CAD-generated  objects  like 
houses,  tanks,  cars)  are  merged  and  converted  into  polygon  data.  At  the  following  step 
tactical  information  of  a  CCIS  is  included  giving  information  about  number,  kind,  and 
starting  position  of  military  units  at  the  field  of  interest.  This  information  is  linked  to 
dynamic  objects  and  included  in  the  final  scene-database  for  visualisation. 

The  second  part  works  online  in  real-time.  Here  the  scene-database  is  displayed  on  a  vir¬ 
tual  workbench  and  interaction  with  the  database  is  possible.  For  post-brief  all  the 
actions  are  written  into  a  protocol  file.  As  the  dynamic  objects  are  linked  to  information 
from  the  CCIS  their  position  may  change  and  the  display  has  to  be  updated. 
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Figure  4:  Brief  structure  of  the  Electronic  Sandtable  (ELSA) 

The  hardware  of  the  systems  consists  of  a  graphic  computer  (Onyx2  Infinite  Rality  2  of 
Silicon  Graphics  Inc.)  and  a  virtual  workbench  with  shutter  glasses.  The  scene-database 
is  created  using  EasyT-Software  by  Coryphaeus  Software  Inc.  and  DIMS  (Digital  Land- 
Mass  System)  data  (AMilGeo,  1996).  The  visualisation  software  uses  IRIS  Performer 
graphics  library  (Silicon  Graphics  Inc.).  With  multichannel-option  one  channel  is  used 
for  the  display  on  a  supervisor’s  monitor  and  the  second  channel  drives  the  workbench  of 
the  Electronic  Sandtable. 

The  virtual  workbench  is  shown  in  figure  5.  It  is  based  on  the  ideas  and  studies  of  Kruger 
(Kruger  &  Frdhlich,  1995).  A  projector  puts  different  frames  for  the  left  and  the  right  eye 
time-multiplexed  via  a  mirror  onto  the  projection  plane.  The  operators  wear  shutter 
glasses,  which  are  synchronized  with  the  projector.  Synchronization  works  by  using  IR- 
emitters  and  IR-sensors  on  the  glasses.  An  interface  for  connectiong  different  VO-devices 
to  the  computer  system  exists. 

The  view  geometry  is  calculated  for  a  single  operator.  His  eye-position  is  assumed  fixed 
at  the  moment  and  perspective  error  grows  with  distance  to  this  position.  An 
achievement  is  expected  by  tracking  the  eye  position  and  updating  the  displayed 
geometry  if  the  position  changes. 

Topics  of  Research 

Since  applicability  and  limitations  of  VE-techniques  in  connection  with  CCIS  are  mostly 
unknown,  there  are  questions  arising  about  the  human  operator  using  VE-interaction  de¬ 
vices  and  VE-TDS. 
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Figure  5: Experimental  facility  of  the  Electronic  Sandtable  (ELSA)  at 

FFM 


Generally  spoken,  interaction  devices  have  to  fulfil  several  ergonomic  requirements  con¬ 
nected  with  the  human  operator’s  abilities  and  limitations.  These  are: 

•  Aspects  of  the  human  perception  with  different  human  senses.  Especially  if 
information  is  displayed  in  a  three-dimensional  way  attention  should  be  paid 
on  depth  perception  of  the  visual  sense; 

•  Aspects  of  the  usability  of  devices  (e.g.  weight,  adaptation  to  all  kinds  of  pro¬ 
tective  clothes)  in  order  to  achieve  a  comfortable  use  for  long  duration. 

Software-ergonomic  requirements  have  to  be  determined  in  order  to  assure  high  situation 
awareness,  acceptable  operator  workload,  and  sufficient  handling  safety  and  reliability 
while  taking  advantage  of  a  three-dimensional  representation  of  the  scenario.  Questions 
on  these  deal  with 

•  adequate  information  presentation, 

•  intuitive  manipulation  of  objects  embedded  in  the  virtual  scene, 

•  ability  for  the  operator  to  build  up  a  mental  model  of  the  system’s 
functionality, 

•  implementation  of  error-tolerant  behaviour  of  the  system, 

•  implementation  of  user  support. 


Applying  VE-techniques  to  a  TSD  brings  along  a  lot  of  questions.  VE-techniques  are 
defined  as  an  "intuitive  perceivable  and  experienceable  scene  of  a  natural  or  abstract 
environment"  (Bullinger  et  al,  1997),  The  software-ergonomic  requirements  are  still 
valid,  but  a  simple  adoption  of  the  results  for  conventional  two-dimensional  computer- 
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interaction  will  lead  to  the  loss  of  nearly  all  benefits  of  VE-systems  This  means  that  new 
ways  of  visualisation  and  interaction  have  to  be  evaluated,  tested  and  optimised. 

The  main  focus  of  research  lies  in  the  following  areas: 

Visualisation:  Human  visual  perception  plays  the  important  role  for  defining  require¬ 
ments  of  future  visual  displays.  Monocular  and  binocular  depth  cues  like  perspective, 
shadowing  or  stereoscopic  parallax  have  different  effects  on  individual’s  depth 
perception.  Studies  about  quantification  of  the  effects  of  modelled  depth  cues  in  virtual 
environment  scenarios  with  a  large  scale  is  not  known.  For  visualisation  of  geographic 
data  a  simple  model  of  the  view  geometry  is  not  enough  because  of  large  scales.  Instead 
view  geometry  has  to  be  modified  in  order  to  create  a  better  depth  effect.  This  will  be 
made  by  using  un-real  eye-to-eye  distances.  This  procedure  is  also  used  for 
photogrametric  or  reconnaissance  purposes. 

Interaction:  A  main  goal  of  VE-Systems  is  the  natural  representation  of  virtual  objects. 
This  leads  to  natural  interaction  with  virtual  objects.  Interaction  in  VE-systems  means 
both:  navigation  in  virtual  worlds  and  manipulation  of  virtual  objects.  Procedures  of 
navigation  in  virtual  environments  and  databases  are  evaluated  and  hardware  devices  are 
tested  with  regard  to  their  practical  use.  Because  view  geometry  of  the  Electronic  Sand- 
table  is  "god’s  eye"  from  direct  above,  navigation  is  roughly  two-dimensional  (horizontal 
and  vertical  scrolling).  This  can  be  controlled  by  conventional  two-dimensional  I/O- 
devices.  Manipulation  of  virtual  objects  (picking,  moving,  etc.)  requires  interaction  with 
more  degrees  of  freedom  (normally  6  DOF).  Because  of  this  6-DOF  interaction  devices 
like  the  space  mouse  or  virtual  pointers  will  be  used. 

Co-operation:  A  multi-user  VE-TSD  offers  the  possibility  of  co-operative  work  and 
virtual  discussions  at  military  staffs.  From  the  technical  point  of  view  multi-user  systems 
bring  along  new  problems.  For  example,  view  geometry  has  to  be  calculated  for  more 
than  one  eyepoint,  which  reduces  the  framerate  of  time-multiplexed  display  systems. 
Multi-user  interaction  techniques  have  only  roughly  been  developed  and  simple  problems 
like  giving  an  object  to  another  user  brings  along  a  lot  of  problems  for  software 
developers.  Another  question  is  whether  all  users  should  have  the  same  interaction 
abilities.  Is  it  better  if  each  user  has  the  same  rights  to  manipulate  objects  or  should  there 
be  a  hierarchy  of  manipulation  rights?  These  topics  are  of  major  interest  for  TSDs  as 
there  is  a  military  hierarchy  in  reality  which  should  have  effects  on  virtual  command 
hierarchy. 

Distribution:  Yet  co-operation  is  only  possible  if  staff  members  meet  at  the  same  place. 
By  use  of  distributed  simulation  this  might  change.  Real  persons  will  meet  in  a  virtual 
environment  as  representations  (so  called  avatars)  and  have  their  staff  meeting  virtually. 
This  future  vision  sounds  veiy  promising,  but  do  remember  that  co-operation,  discussion 
and  brainstorming  meetings  are  events  in  the  real  world.  There  are  sociologic, 
psychologic  and  individual  factors  involved  in  real  world  which  cannot  be  simulated. 
Because  of  this  the  benefits  of  distributed  TSDs  have  to  be  quantified  and  examined  very 
carefully. 

Conclusion 

Virtual  Environment  technology  brings  along  a  lot  of  new  ideas  for  military  TSDs. 
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Especially  the  natural  way  of  presenting  complex  data,  the  possibility  of  including  assist¬ 
ing  functions  and  the  high  actuality  when  connected  with  electronic  CCIS  are  huge 
benefits  for  the  use  of  such  systems. 

Nevertheless  the  human  factors  have  to  become  the  main  points  of  interest  to  define 
needs  and  requirements  for  an  integration  of  VE- systems.  If  this  does  not  happen,  the 
benefits  wiU  change  into  disadvantages  and  human  operators  will  not  be  assisted  but 
confused  by  visualisation  of  the  information  data  flow. 
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Abstract 

Modem  multimedia  decision  support  systems  are  characterized  by  continuously  chang¬ 
ing  conditions  regarding  technology,  task  and  user  profiles.  As  a  consequence  of  this 
heterogeneity  a  huge  amount  of  information  items  of  different  data  types  has  to  be  man¬ 
aged  and  processed.  From  a  human  factors  point  of  view  there  is  a  need  to  bridge  the  gap 
between  general  ergonomic  design  principles  (e.g.,  suitability  for  the  problem,  flexibility, 
controllability)  and  concrete  implementations.  Solutions  have  to  be  found  to  provide 
homogeneous  user  interfaces  with  consistent  access  and  evaluation  capabilities  for  dif¬ 
ferent  data  types.  Another  problem  is  the  dynamic  aspect  of  information  flow.  One  has  to 
take  into  consideration  that  -  especially  in  hierarchical  management  information  systems 
(MIS)  -  information  items  may  change  their  data  type  during  processing  steps.  E.g„  a 
broad  range  of  textual  input  details  may  be  aggregated  and  transformed  to  other  presen¬ 
tation  categories  to  support  decision  processes  at  a  higher  management  level.  To  achieve 
those  goals  integrated  information  systems  require  problem  solutions  depending  on 
document-oriented  working  styles  instead  of  traditional  program-oriented  solutions. 

In  client/server  environments  the  data  management  concept  should  be  developed  inde¬ 
pendently  from  data  types  used  in  the  operational  system.  It  can  be  stated  that  good, 
clear  process  models  are  the  key  to  design  the  core  of  an  information  system.  Efficient 
CASE  technology  allows  for  breaking  down  the  major  processes  into  lower-level  proc¬ 
esses  step  by  step.  During  this  top-down  process  software  generators  may  be  used  to  cre¬ 
ate  applications  and  interfaces  with  respect  to  ergonomic  styleguides  to  ensure  consistent 
interaction  techniques  even  in  multimedia  applications.  Another  approach  to  meet  the 
ergonomic  requirements  in  complex  MIS  is  to  make  use  of  modem  object-oriented  con¬ 
cepts  (e.g„  COM,  CORBA).  This  takes  into  account  that  nowadays  the  ability  to  inte¬ 
grate  different  generic  software  components  is  of  greater  importance  than  developing 
specific  applications.  Several  basic  principles  to  achieve  this  integration  will  be  ad¬ 
dressed  in  this  paper. 

Introduction 

Our  research  efforts  are  directed  towards  the  development  and  application  of  human  en¬ 
gineering  methods  and  criteria  to  the  design  and  evaluation  of  complex  man-machine 
systems  in  all  phases  of  system  development  and  acquisition.  Multimedia  communica¬ 
tion  is  ubiquitous  in  those  domains.  When  humans  converse  one  another,  they  utilize  a 
wide  range  of  media  to  interact.  Some  media  of  communication  are  more  effective  than 
others  for  certain  situations,  users  or  contexts.  A  multimedia  document  which  includes 
text,  tables,  graphics,  speech,  video,  may  effect  several  human  senses  to  process  the  in¬ 
coming  information.  This  interaction  occurs  over  time.  Therefore,  it  is  necessaiy  to  ac¬ 
count  for  the  processing  of  discourse,  context  shifts,  and  changes  over  time.  Whereas 
humans  have  a  natural  facility  for  managing  and  exploiting  multiple  input  and  output 
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media,  computers  do  not.  Human  abilities  should  be  amplified  by  using  computers,  and 
the  synergistic  utilization  of  multiple  media  can  support  this  amplification.  If  appropriate 
media  are  utihzed  for  human  computer  interaction,  there  is  a  potential  to  increase  the 
bandwidth  of  information  flow  between  human  and  machine.  To  achieve  this  goal  re¬ 
quires  a  deep  understanding  of  information  characteristics,  how  they  relate  to  character¬ 
istics  of  media,  and  how  they  relate  to  models  of  taisks,  users,  and  environments.  Under¬ 
standing  these  principles  will  not  only  result  in  better  interactive  devices,  but  also  leads 
to  new  tools  for  context-sensitive  multimedia  interfaces,  and  intelligent  agents  for  multi- 
media  information  retrieval,  processing,  presentation,  and  authoring. 

Two  approaches  dealing  with  multimedia  information  processing  are  covered  in  this  pa¬ 
per.  First,  it  is  illustrated  how  actual  tendencies  in  software  development  technologies 
can  be  utilized  to  design  user-  and  task-oriented  multimedia  systems  which  allow  for 
document-oriented  working  styles.  Second,  it  is  shown  how  integrated  CASE-technology 
can  be  applied  to  meet  general  ergonomic  requirements  in  complex  database  manage¬ 
ment  systems. 

A  modular  workstation 

Modem  workstations  designed  for  complex  information  systems  must  support  a  wide 
range  of  functionalities.  In  our  application  domain  these  are  military  situation  display, 
geographic  information  processing  as  well  as  text  processing,  spread  sheet  applications 
and  business  graphics.  The  incorporation  of  video  and  audio  will  be  standard  in  the  near 
future  to  realize  multimedia  presentations  for  situation  reports.  Those  workstations  are 
embedded  in  heterogeneous  communication  networks  where  the  combined  use  of  com¬ 
mercial,  government,  and  military  software  products  (CGM-software)  is  ubiquitous. 
Workstation  design  means  to  use  and  integrate  specific  characteristics  of  different  prod¬ 
ucts  which  can  be  considered  as  independent  „components“.  In  such  a  modular  environ¬ 
ment  state  of  the  art  is  a  program-oriented  workstyle.  This  means  that  operators  use  dif¬ 
ferent  tools  for  different  tasks.  The  aim  is  -  however  -  to  construct  homogeneous  user 
interfaces  based  on  complex  documents  which  consist  of  different  building  blocks  proc¬ 
essed  by  distinct  tools. 


situation  /  map 
spread  sheet 


audio 


business  graphics^^ 
text  proems  in  g 


animation 


A  modular  workstation  for  command  and  control  information  systems 
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ModeUVieW’Controller 

In  order  to  use  efficient  tools  for  the  management  of  multimedia  data  the  before  men¬ 
tioned  „components“  should  not  be  responsible  for  data  storage  and  organization.  Ac¬ 
cording  to  the  ,4nodel“View-controller“-concept  (MVC)  known  from  Smalltalk,  informa¬ 
tion  container  should  be  realized  in  separate  databases.  This  separation  of  multimedia 
database  (model)  and  user  front  ends  (views)  gives  the  chance  to  implement  powerful 
algorithms  for  data  evaluation  and  aggregation  across  different  data  types.  These  algo¬ 
rithms  can  be  implemented  independently  from  specific  user  interfaces  and  can  be 
adapted  to  actual  task  requirements.  On  the  other  hand  ,4ntelligent“  agents  can  be  devel¬ 
oped  to  interpret  user  needs  and  control  the  information  flow  between  database  and  user 
front  end(s).  E.g.,  a  situation  display  can  present  data  on  a  geographical  map  as  well  as  in 
a  textual  form  or  in  a  tree  structure  if  appropriate.  While  the  model  represents  the  actual 
situation  in  a  formal  description,  different  views  can  be  applied  to  present  the  data  in  a 
task-oriented  way. 


System  architecture  with  separate  modules:  (1)  views,  (2)  model,  (3)  controller 


Compound  document 

The  mentioned  MVC-concept  gives  the  freedom  to  implement  different  accesses  to  the 
various  data  sources.  The  aim  is  to  design  a  task-oriented  man-machine-interface  without 
overloading  the  operator  with  a  huge  amount  of  functions  of  different  programs,  A  solu¬ 
tion  is  the  integration  of  objects  in  a  single  document  to  ensure  that  the  user  can  concen¬ 
trate  on  his  work.  Object  Linking  and  Embedding  (OLE)  is  a  method  used  by  Micro¬ 
soft’s  Windows  products  to  integrate  the  output  of  one  program  (server  task)  as  data  into 
another  (client  task)  -  for  example,  a  drawing  into  a  word  processing  document.  The  ap¬ 
propriate  server  program  is  automatically  run  to  edit  the  data  when  the  OLE-object  is 
activated.  The  newer  version  2.0  of  OLE  supports  in-place-editing,  so  that  rather  than 
opening  or  moving  to  a  new  window,  it  lets  you  edit  the  embedded  object  without  hiding 
the  rest  of  the  document.  Clicking  starts  the  server  application  and  changes  only  menus 
and  toolbars.  Competing  approaches  like  Apple’s  OpenDoc  and  IBM’s  System  Object 
Model  SOM  work  in  a  similar  manner.  The  difference  between  Object  Linking  and  Em¬ 
bedding  and  conventional  copy-and-paste-strategies  is  that  with  OLE,  the  data  may  be 
edited  within  the  client  document.  Documents  that  contain  links  to  the  output  from  other 
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applications  are  called  ^compound  documents“.  The  underlying  „Componentware“- 
technique  can  be  applied  to  configure  the  user  interface  according  to  actual  situation 
specific  needs.  The  figure  shows  such  a  compound  document  consisting  of  portions  like 
map  display,  table,  picture,  graphics  and  embedded  video  sequence. 


Compound  document  as  integrated  interface  to  dijferent  tools 


OLE  allows  to  initiate  a  program-to-program-connection  manually.  Furthermore  is  it 
possible  to  create  automatic  connection  between  application  by  using,  e.g.,  OLE  custom 
controls  (OCX).  The  link  from  one  application  to  another  is  installed  permanently  and 
can  be  activated  whenever  the  user  wants  to  do  this.  These  automatic  links  may  be  used 
to  combine  the  features  of  different  programs  in  a  given  context.  E.g.,  a  display  system 
may  be  applied  to  present  a  situation  in  front  of  a  geographic  map.  Using  OCX- 
techniques  an  agent  may  be  instructed  to  read  additional  background  information  from  a 
database  and  present  this  text  in  word  processing  program.  Other  techniques  called  „ vis¬ 
ual  programming“  may  be  used  to  configure  a  graphical  user  interface  independently 
from  data  processing  components.  Those  components  can  be  linked  dynamically  to  inter¬ 
face  commands  such  as  menus  items,  toolbars  and  buttons  to  invoke  the  required  func¬ 
tions. 

Multimedia  database  applications 

In  modem  client/server  environments  the  mass  of  information  is  stored  in  central  ar¬ 
chives.  CASE  technology  (computer  aided  systems  engineering)  can  support  the  process 
of  generating  large  databases  and  complex  applications  based  on  multimedia  data  sets. 
Graphical  design  tools  assist  developers  defining  the  application  domain  by  process  and 
system  models.  The  description  is  hold  in  a  central  repository.  Automatic  generators  cre¬ 
ate  basic  prototypes  with  consistent  look  and  feel  as  well  as  consistent  behavior  of  the 
application  system.  The  automatic  generation  process  can  be  controlled  by  supporting 
components  such  as  libraries,  preferences,  master  forms,  and  templates.  This  can  be  used 
to  direct  the  system  generation  in  a  way  that  ergonomic  know-how  is  implicitly  incorpo¬ 
rated  in  the  design  process.  Libraries  define  the  functionality  (procedures)  for  different 
user  interfaces  in  a  consistent  manner.  Preferences  specify  general  adjustments  about  the 
layout  of  applications,  like  window  behavior.  Templates  hold  exact  information  about 


common  parts  of  the  layout,  color  settings,  etc.  Master  forms  are  used  in  an  object- 
oriented  way  to  develop  application  specific  forms. 

Because  routine  work  is  performed  by  CASE  tools,  the  system  developer  can  concentrate 
on  the  design  of  the  interactive  user  interface.  According  to  the  MVC-concept  this  inter¬ 
face  is  not  directly  influenced  by  the  central  data  management  system.  Therefore  it  is 
possible  to  develop  different  interactive  access  components  based  on  the  same  data 
model.  The  figure  gives  an  example  to  illustrate  the  development  process  from  „user  re- 
quirements“  to  ,4nultimedia  applications^  based  on  CASE  technology: 

user  requirements  are  analyzed  and  modeled  in  a  formal  language 
process  and  system  model  are  managed  in  the  central  repository 
additional  components  control  the  system  generation  process  in  a  predefined  way 
interface  prototypes  are  optimized  to  ensure  an  optimized  presentation  of  data 
(e.g,,  tabular  views  to  display  data  records,  graphic  views  to  show  maps, 
videos,..) 


CAS  E-technology  used  to  bridge  the  gap  between 
general  ergonomic  requirements  and  actual  systems  design 


Conclusions 

Flexible  and  intelligent  multimedia  interfaces  promise  to  enable  systems  and  people  to 
use  media  to  their  best  advantage.  They  can  increase  the  bandwidth  of  information  flow 
between  human  and  machine.  New  technologies  can  help  to  solve  the  structural  problems 
coming  along  with  the  combined  use  of  enriched  media.  They  can  also  help  that  human 
abilities  are  amplified,  not  impeded,  by  using  computers.  The  introduction  of  a  docu¬ 
ment-oriented  working  style  helps  that  users  can  concentrate  on  their  goals.  Controlling 
the  software  generation  process  enables  new  facilities  to  incorporate  ergonomic  know¬ 
how  into  the  design  procedure. 
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Abstract 

Modem  warfare  requires  a  comprehensive  understanding  of  effective  decision  making, 
clear-cut  command  and  control  procedures,  and  high-capacity  communication  systems. 
At  the  tactical  and  lower  unit  levels  this  understanding  is  decisive,  because  lack  of 
insight  in  these  issues  can  cause  severe  consequences  for  the  intended  outcome  of  future 
operations,  and  ultimately,  unnecessary  loss  of  life. 

In  this  work,  we  developed  a  framework  for  analysis,  modeling,  and  evaluation  of  the 
combat  skills  of  low-level  ground  forces,  commander  combat  resource  management,  and 
overall  unit  performance.  This  framework  originated  from  various  sciences:  control 
theory,  quahty  control,  certain  behavioral  and  human  factors  areas,  operations  research, 
and  statistics.  The  integration  of  these  well-  proven  research  areas,  together  with 
advanced  high-performance  ground  combat  simulation,  into  a  novel  concept  of  Combat 
Efficiency  Analysis  (CEA),  facihtated  simulation-supported,  unambiguous  and 
comprehensive  military  unit  evaluation  and  assessment. 

Two  full-scale  studies  of  mechanized  infantry  units  against  a  realistic  adversary  were 
carried  out.  The  subsequent  CEA  provided  a  powerful  aid  for  understanding  combat 
essentials  and  for  requirements  generation  based  upon  hard  facts.  The  described  analysis 
and  modeling  approach  made  it  possible  to  identify  various  limiting  factors  for  combat 
success  of  ground  forces. 

Introduction 

Unclassified  research,  regarding  the  specific  kinds  of  skills  and  properties  that  is  needed 
for  optimum  performance  at  the  battalion,  company  and  platoon  levels  of  modem  tactical 
forces,  has  been  rarely  published.  The  striking  properties  of  such  forces  can  be 
characterized  in  brief  as  constantly  increasing  mobility,  lethality  and  level  of  protection. 
Corresponding  enhancements  are  made  concurrently  in  tactics  and  intelligence.  These 
dynamic  properties  raise  a  demand  for  novel  solutions  to  several  of  the  eternal 
warfighting  problems:  Individual  and  team  situation  awareness  and  decision  making, 
commander  and  soldier  combat  resource  management,  and  combat  endurance  of  the 
units  and  systems  engaged  in  the  mission.  Determined  and  forward  exploitation  and 
control  of  these  belligerent  system  dynamics  are  vital  for  success.  Soldier/operator  and 
commander  skills  in  managing  and  mastering  these  dynamics  have  decisive  impact  on  all 
decisions  and  selections  of  action,  the  battle  course  of  events,  logistics,  the  number  of 
casualties,  and  many  other  vital  components  of  war  or  severe  crisis. 
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Worm  (1996)  developed  a  set  of  methods  and  tools  for  modeling,  analysis,  and 
evaluation  of  ground  forces  and  their  abilities  at  the  battalion  and  lower  levels:  Combat 
skills,  commander  combat  resource  management,  and  overall  unit  performance.  The 
central  point  of  this  project  was  the  integration  of  a  number  of  methods  and  tools,  used 
for  many  years  in  trade  and  industry  as  well  as  in  military  systems  development,  into  a 
multi-discipline  mission  and  unit  evaluation  and  assessment  technique.  To  facilitate  this 
integration,  a  set  of  concepts  were  introduced  in  order  to  analyze  and  evaluate  the 
accomplishments  and  shortcomings  of  various  military  units  in  an  unambiguous  and 
comprehensive  way.  The  concepts  were: 

•  Control  theory-based  conceptual  modeling  of  dynamic  and  complex  combat 
systems  and  processes,  and  of  their  states  and  state  changes. 

•  Identification  of  mission  and  unit  state  variables,  and  of  different  action  and 
decision  making  mechanisms  as  a  combat  process  regulator. 

•  Within-mission  Combat  Efficiency  Analysis  of  fully  manned  and  equipped 
company  units  against  a  realistic  opposing  force. 

The  rest  of  this  paper  is  set  out  as  follows,  hi  the  next  four  sections  we  describe  the 
development  of  a  dynamic  system  model  of  tactical  missions.  The  following  section 
elaborate  on  mission  and  unit  state  variables  identification,  and  on  regulating  combat 
processes.  The  last  sections  describe  the  Combat  Efficiency  Analysis  technique  applied 
in  a  case  study.  Conclusions  and  an  outline  of  future  work  concludes  the  paper, 

A  Dynamic  System  Model  of  Tactical  Missions 

By  system  is  meant,  in  control  theory,  an  object,  driven  by  external  input  signals  u(t)  for 
every  t  and  as  a  response  produces  a  set  of  output  signals  y( t)  for  every  t.  From  the  work 
of  Ashby  (1956),  Brehmer  (1992),  and  Ljung  and  Glad  (1995),  it  is  well  known  that  most 
complex  systems  have  real-time,  dynamic  properties,  i.e.  the  value  of  the  output  at  a 
given  time  is  not  only  dependent  of  the  input  value  at  this  specific  time,  but  also  on 
earlier  input  values.  The  difficulties  of  maintaining  situation  awareness  in  such  dynamic 
environments  cause  specific  problems.  Endsley  (1995)  developed  a  comprehensive 
theory  of  individual  operator,  commander,  and  team  situation  awareness  in  dynamic 
environments.  In  this  work,  an  extended  system  terminology  was  used.  In  the  domain  of 
modem  and  future  military  combat,  and  military  command  and  control.  Worm  (1996) 
defined  three  main  system  components: 

1.  Technological  Systems,  e.g.  weapons,  vehicles,  communication-,  reconnaissance-, 
decision  support-,  or  command  support  systems,  along  with  system  operators. 

2.  Command  and  Control  Systems,  i.e.  information  structures,  command  structures, 
and  decision  stmctures,  built  up  by  technological  systems,  intelligence  acquisition 
systems  and  directly  involved  decision  makers,  arranged  in  command  hierarchies  and 
chains  of  command.  A  few  examples  of  command  and  control  systems  are: 

•  Tactical  Military  Command  and  Control  Systems 

•  Fire  Support  Command  and  Control  Systems 

•  Emergency  Management  Command  and  Control  Systems 

•  Logistic  Command  and  Control  Systems 

•  Transportation  Command  and  Control  Systems 
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3.  Units,  i.e.  organizational  aggregates,  whose  structures  consistof  one  or  several 
technological  systems,  soldiers/operators,  commanders,  command  and  control 
systems,  staff  functions,  support  functions,  and  other  special  services. 

The  Mission  Control  Problem:  Definitions  and  Properties 

According  to  Ljung  and  Glad  (1995),  modem  control  theory  formulated  the  control 
problem  as  follows: 

” Given  a  system  (S),  with  available  measurement  (y),  determine  the  controller  output 
(u),  so  that  a  control  signal  (z)  follows  a  reference(r),  despite  the  influence  of  external 
disturbances  (w),  measurement  errors  (n),  and  system  variability,  while  keeping 
controller  output  values  within  reasonable  limits.  ” 

This  definition  is  illustrated  in  figure  1. 


The  Basic  Concepts  of  a  Generic  Combat  Mission  Model 
The  System  is  the  own  military  and/or  civilian  unit,  defined  above. 

The  Controller  Output  to  the  system  consists  of  a  flow  of  information.  The  information 
components  are  orders,  intelligence,  and  other  control  components,  transmitted  via 
available  communication  systems. 

The  System  Reference  is  constituted  by  the  missions,  objectives,  goals,  and  the  required 
system  status  of  the  unit. 

The  System  Process  is  in  this  case  the  battle  which  is  strongly  dynamic  and  non-linear, 
i.e.  changes  in  the  process  can  give  non-proportional,  unpredictable  or  even  chaotic 
consequences. 

The  Combat  Environment  (i.e.  the  enemy,  his  systems,  the  terrain,  and  other  battlefield 
components),  and  its  impact  on  the  own  unit  and  the  combat  conditions,  are  treated  as 
disturbances. 
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The  Controller  compensates  for  the  disturbances.  The  controller  function  consists  of  the 
planning,  decision  making,  orders,  and  actions  of  commanders  and  soldiers  within  the 
own  unit. 

Measurement  are  all  values  that  can  be  registered,  measured,  and  processed.  There  are 
also  Measurement  Error  that  must  be  handled.  The  Measurement  Error  consists  of  a 
number  of  components,  of  which  two  of  the  most  important  was  identified  this  far 
(Worm,  1996).  Those  are  Incorrect  or  Insufficient  Situation  Awareness^  and  Insufficient 
or  Misleading  Intelligence. 

The  System  Output  is  the  Influence  on  Battle  Course  of  Events^  and  Influence  on  Enemy 
Decisions,  Actions,  and  Planning. 

Feedback:  If  you  use  the  system  output  to  determine  the  system  input  (closed-loop 
control,  feedback  control),  there  is  only  a  limited  need  for  knowledge  on  system 
dynamics.  You  can  then  continuously  make  necessary  adjustments  by  measuring  the 
deviation  of  the  system  output  from  the  reference  value. 

The  Combat  Mission  Model 

Combat  missions  can  be  described  as  clusters  of  processes,  arranged  in  a  dynamic  system 
model  aggregate,  depicted  in  figure  2, 


Figure  2:  A  simple  combat  model 


Description  and  Determination  of  Mission  and  Unit  States 

With  the  System  State  at  a  given  time  is  meant  a  set  of  information,  e.g.  variable  values 
that  makes  it  possible  to  determine  future  system  output  if  future  system  input  is  known. 

Before  the  battle,  the  unit  has  an  initial  unit  state  that  is  continuously  affected  by  all 
actions  and  events  occurring  in  the  combat  environment.  Correspondingly,  the  mission 
has  an  initial  mission  state  that  is  influenced  by  all  maneuvers  and  outcomes  of  the 
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coming  battle.  When  the  activity,  function,  status  or  mission  objective  of  the  unit 
changes,  due  to  any  cause,  a  state  change  takes  place.  Initial  unit  and  mission  states  are 
primarily  determined  by  the  following  factors: 

1.  The  status  of  soldiers  and  operators  concerning: 

•  individual  and  team  skills 

•  combat  experience 

•  physical  capacity  and  injuries 

•  psychological  resistance  to  trauma  and  stress 

•  psychosocial  capacity 

•  motivation 

2.  Commander’s  support  and  improvement  of  unit’s  status  concerning: 

•  confidence  in  subordinate  commanders 

•  fostering  a  good  unit  spirit 

•  practicing  and  encouraging  good  leadership 

•  good  personal  judgment 

•  caring  for  subordinated  personnel 

•  evaluating  experienced  combat  situations  together  with  all  personnel  involved 

•  assigning  appropriate  mission  tasks 

•  continuous  training  and  adaptation  of  mission  tactics  and  procedures 

•  well-functioning  maintenance,  logistics,  and  medical  services  within  the  unit 

•  sustaining  good  physical  and  psychological  performance 

3.  Materiel  and  ordnance  status  concerning 

•  technical  availability 

•  tactical  availability 

•  damage  assessment  and  maintenance  requirements 

•  depleted  resources 

•  replenished  resources 

4.  Information  status  concerning 

•  operations  orders  and  mission  objectives 

•  the  reliability,  availability  and  diagnosticity  of  intelligence  and  information 

•  access  to  and  utilization  of  intelligence/information  acquisition  resources 

•  reporting  from  own  and  adjacent  units 


Case  Study:  Mechanized  Infantry  Combat  Efficiency  Analysis 

The  integration  of  the  system  models,  concepts,  and  components  above  now  leads  to  the 
definition  of  the  combat  efficiency  measure,  that  will  be  applied  in  the  case  study  in  the 
following  sections.  The  definition  of  combat  efficiency  and  its  determinants  is  depicted 
in  figure  3,  next  page. 
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Figure  3:  Definition  and  determinants  of  the  combat  efficiency 
measure. 


The  Mission:  Attack  Enemy  Airborne  Assault 

In  this  case  of  unit  evaluation  and  assessment,  the  unit  at  study  was  a  reduced 
mechanized  infantry  company,  transported  in  armored  personnel  earners,  and  equipped 
with  man-portable  anti-tank  missile  systems  as  its  main  armament.  The  unit  also  had 
access  to  indirect  fire  support,  i.e,  artillery  and  mortars,  and  anti-tank  mines.  The 
mission  of  the  mechanized  infantry  company  were  to  locate  and  attack  enemy  airborne 
landings  in  either  of  three  geographical  sectors,  designated  sector  A,  B,  and  C. 

The  opposing  force  in  this  particular  situation  was  a  special  unit,  organized  and 
composed  by  uniquely  equipped  and  trained  mechanized  platoons,  with  tactics,  mobility, 
and  firepower  equal  to  a  hypothetical,  modem,  enemy  air  assault  unit.  The  mission  of 
the  air  assault  forces  were  to  seize  and  defend  a  number  of  vital  crossing  points  in  order 
to  support  following  airborne  landings. 

The  combat  environment  and  the  effects  of  major  weapon  systems  were  simulated  in  the 
MIND  simulation  environment,  (Jenvald,  Morin,  Worm,  and  Omberg,  1996).  The 
MIND  system,  which  is  an  integrated  simulation  and  data  collection  system  for 
instrumented  foice-on-force  battle  training,  is  currently  used  by  the  Swedish  Army  in  its 
Combat  Training  Center,  at  which  the  described  case  study  took  place. 

The  Combat  Efficiency  Analysis  was  based  on  aggregation  and  evaluation  of  expert 
observations,  registered  combat  events,  and  After  Action  Reviews  (Rankin,  Centner,  and 
Crisscy  (1995)  of  two  consecutive  battle  training  sessions,  in  which  the  mission  and 
combat  requirements  were  close  to  identical.  The  participating  units  were  the  same,  and 
the  operations  orders  were  adjusted  to  compensate  for  learning  and  transfer  effects. 
Information  lost  (if  any)  due  to  technical  failure  were  not  able  to  take  into  consideration. 
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but  judging  from  the  consistency  of  the  results  of  the  two  sessions,  this  fact  did  not  affect 
the  data  compilation  and  analysis  in  any  decisive  way. 

Initial  Mission  and  Unit  States 

Status  of  Soldiers  and  Operators:  Individual  and  team  skills  were  limited  because  basic 
training  was  not  completed;  only  6,5  months  of  a  total  of  10  to  12  months  were  earned 
out.  This  induced  limitations  in  the  unit’s  ability  to  correctly  handle  unforeseen 
situations,  not  yet  experienced  by  the  unit,  and  this  was  taken  into  consideration  in  the 
analysis.  The  fact  that  the  unit  was  slightly  reduced  caused  no  significantly  limited 
availability  of  personnel  and  materiel  in  battle  training  sessions  as  short  as  in  this  study. 

Commander  Sustainment  and  Improvement  of  Unit's  Status:  Main  limiting  factors  were 
that  commanders  had  not  yet  performed  any  evaluation  of  experienced  combat  situations 
together  with  all  personnel  involved  and,  consequently,  were  not  able  to  train  and  adapt 
mission  tactics  and  procedures  associated  with  the  battle  to  come. 

Materiel  and  Ordnance  Status:  Unit  had  good  logistic  preparedness.  No  materiel 
lacked,  and  supplies  of  fuel  and  ammunition  were  full. 

Information  Status:  Tactical  operations  orders  were  understood  and  disseminated  into 
clear,  straightforward  operations  orders.  Preparations  orders  were  issued  and  executed. 
The  limiting  factor  was  terrain  knowledge,  and  also  intelligence,  which  was  uncertain, 
unconfinned,  and  several  hours  old. 

Battle  Course  of  Events 

The  outline  of  the  mission,  preparations,  enemy  approach  to  landing  zone  and  acquired 
intelligence  prior  to  engagement  are  briefly  depicted  in  the  mission  timeline  in  table  1. 
From  the  time  of  landing  and  engagement  of  the  air  assault  forces  the  course  of  events 
were  followed  by  expert  observers  and  controllers,  most  of  which  were  experienced 
combat  training  officers  in  active  duty.  They  reported  combat  events  supported  by  a 
computerized  tool  for  structured  activity  reporting  (Jenvald  and  Morin,  1997),  which 
allowed  automated  processing  of  the  very  large  quantity  of  information  acquired. 

Battle  course  of  events  were  violent  and  expeditious.  Approximately  sixty  minutes  from 
the  landing  of  the  assault  forces  at  time  T,  the  combat  activity  culminated  and  the  session 
stopped  sixty  minutes  later,  in  order  to  let  both  sides  handle  disengagement  and 
preparations  of  reengagement,  and  to  take  care  of  the  simulated  casualties  and  wounded, 
thereby  increasing  workload  of  the  logistics  function.  At  the  After  Action  Review, 
commanders  and  soldiers  of  both  sides,  together  with  the  mission  training  officers, 
concluded  that  the  opposing  air  assault  forces,  despite  large  losses  at  one  flank,  were  able 
to  break  through  or  avoid  the  mechanized  infantry  units  at  the  other  flank,  and 
accomplish  their  mission. 
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Table  1:  Mission  Timeline 


Time 

Own  unit  events 

Opposing  force  events 

Time 

1900 

Session  start. 

Tactical  operations  orders  issued: 
Prepare  location  and  attack  of 
enemy  airborne  assault  in  sector 

A,  B,  or  C. 

Session  start. 

Tactical  operations  issued: 

Prepare  airborne  assault  in  sector 
A,  B,  or  C. 

1900 

0600 

Company  operations  orders 
issued:  Locate  and  attack  enemy 
airborne  assault  in  sector  A,  B,  or 
C. 

Company  operations  orders 
issued:  Commence  airborne 
assault  in  sector  A. 

Bomber  sorties  launched. 

0500 

0630 

Air  defense  warning  issued:  Air 
attacks  east  Skovde  city,  sector  A 
and  B. 

Assault  preparations.  Air  strikes 
in  sector  AandB.  Assault  unit 
boarding. 

1 

Air  defense  warning  issued:  Air 
attacks  Karlsborg  city  westward, 
sector  A  and  B. 

Last  bomber  sorties  launched. 
Assault  forces  launched. 

■ 

Air  defense  warning  issued:  Air 
assault  Karlsborg  city  heading 
Skovde  city. 

Assault  forces  approach  landing 
zone  in  sector  A. 

■ 

0725 

Battlefield  intelligence  reports: 

Air  assault  landing  in  sector  A. 

Time  T:  Assault  units  landing. 

0725 

Indirect  fire  support  authorized. 

Assembly  of  units  after  landing. 

0745 

0800 

Maneuvering  towards  enemy 
location. 

Visual  contact.  Engagement. 

Time  T  -f  35  minutes:  Advance 
towards  target  zone. 

Engagement. 

0800 

1030 

Session  stops. 

Preparations  for  After  Action 
Review. 

Session  stops. 

Preparations  for  After  Action 
Review. 

1030 

Information  processing  during  the  battle 

The  flow  of  information  and  intelligence  within  and  between  the  units  engaged  during 
the  session  indicated  grave  inaccuracy  and  misconception.  Intelligence  was 
communicated  slowly  between  the  different  levels  of  command,  and  was  not  properly 
acquired,  compiled,  and  interpreted.  This  led  to  information  time  delays  that  caused  the 
mechanized  infantry  company  commander  to  have  an  incomplete  intelligence  support  in 
his  situation  assessment,  and  hence,  his  situation  awareness  deteriorated. 
Communication  systems  caused  information  loss  due  to  bad  sound  quality,  units  out  of 
range  or  partial  technical  function  losses. 

Decision  making  and  actions  during  the  battle 

Altogether,  the  insufficient  intelligence  support  and  the  communications  difficulties 
constituted  the  major  part  of  the  inconsistencies  in  the  unit’s  shared  situation  awareness, 
and  the  inaccuracies  and  misjudgments  of  the  mechanized  infantry  company 
commander’s  decisions.  The  company  commander  did  not  realize  the  need  for 
reconnaissance  patrols  for  potential  enemy  sector  surveillance,  nor  did  he  identify  the 
necessity  of  indirect  fire  forward  observers  in  the  potential  landing  sectors  for  immediate 
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response  to  enemy  actions.  Both  of  which  showed  later  to  be  of  crucial  importance  to  the 
fulfillment  of  the  mission.  This  led  to  feedback  delays  in  decision  making  and  to  actions 
that  caused  severe  consequences  for  the  unit’s  ability  to  fight  in  a  coordinated  and  timely 
manner.  The  results  were  consistent  with  Brehmer  and  Allard  (1991),  who  addressed  the 
issue  of  time  delays  in  real  time  decision  making. 

Utilization  of  available  resources  during  the  battle 

During  the  battle  training  sessions  the  mechanized  infantiy  company  had  an  almost 
unlimited  amount  of  simulated  indirect  fire  support  available,  with  capability  of 
combating  both  soft  and  hard  targets.  Much  of  the  indirect  fire  was  unobserved,  which 
led  to  low  hit  probabilities  and  waste  of  exclusive  fire-and-forget  anti-tank  ammunition. 
A  great  number  of  anti-tank  mines  were  hardly  used  at  all  in  the  battle,  which  caused 
difficulties  in  blocking  terrain  not  possible  to  cover  with  troops.  The  units  used 
unnecessary  short  firing  distances  of  their  medium-range  anti-armor  weapon  systems. 
The  units  also  primarily  used  firing  positions  towards  the  enemy’s  front.  All  this  caused 
low  terminal  weapon  effects,  low  effects  of  surprise  and  of  holding  a  well-prepared  firing 
position,  and  high  risks  of  detection  and  counter-attack,  with  large  losses  as  a 
consequence.  The  mechanized  infantry  company  did  not  accomplish  its  mission,  to 
efficiently  locate  and  attack  air  assault  forces  within  its  perimeter.  The  unit  suffered 
from  gravely  reduced  combat  sustainment  and  reengagement  capability  after  the  first 
engagement.  This  rendered  the  mechanized  infantry  company  low  combat  efficiency. 


Conclusions 

The  integration  of  various  well-proven  research  domains,  such  as  control  theory,  several 
behavioral  and  human  factors  areas,  quality  control,  operations  research,  and  statistics, 
together  with  extensive  utilization  of  the  MIND  system,  turned  out  to  be  a  successful 
approach.  Applied  in  a  context  of  ground  force  battle  training,  combined  with  advanced 
medium  and  high  fidelity  combat  simulations,  and  the  systematic  use  of  domain  expert 
knowledge,  this  novel  approach  facilitated  comprehensive  militaiy  unit  evaluation  and 
assessment.  From  the  performed  Combat  Efficiency  Analysis  we  concluded,  that  in  this 
particular  case,  the  following  factors  limited  and  constrained  the  unit’s  ability  to  execute 
its  mission  with  its  available  resources: 

•  The  ability  to  rapidly  and  accurately  determine  the  identity  and  location  of 
enemy  targets  and  of  own  units,  and  to  evaluate  the  battlefield  terrain. 

•  The  access  to  and  use  of  a  battle  information  structure  which  supports  and 
improves  real  time  information  and  intelligence  acquisition,  and  permits 
mission-relevant  information  and  intelligence  to  reach  the  intended  decision 
maker  in  a  timely  manner. 

•  The  access  to  and  use  of  robust,  wide-band  communication  systems  that 
permit  fast  and  accurate  transfer  of  data  and  speech. 

To  be  able  to  objectively  and  reliably  identify  the  hmiting  factors  of  a  specific  unit, 
system,  or  operating  procedure,  and  to  assess  the  magnitude  of  influence  of  these  factors 
on  unit’s  overall  combat  performance,  a  series  of  within-mission  Combat  Efficiency 
Analyses  in  will  have  to  be  performed  in  each  typical  case.  In  force-on-force  battle 
training  situations,  most  factors  except  the  ones  studied  can  be  held  constant  to  the 
greatest  extent  possible,  which  makes  the  validation  of  results  easier.  Also  actual  combat 
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missions  can  be  analyzed  in  the  same  way.  However,  the  extreme  risk  exposure,  and  the 
reduced  reliability,  diagnosticity,  and  availability  of  information  and  intelligence 
obtained  in  armed  conflict  or  other  hostile  situations,  makes  validation  more 
cumbersome. 

Future  Work 

There  are  several  areas  in  which  further  development  and  refinement  of  the  Combat 
Efficiency  Analysis  technique  could  be  of  vital  importance.  A  few  examples  are: 
military  training  (especially  training  tactical  decision  making  and  combat  resource 
management),  battlefield  intelligence  (using  the  Combat  Efficiency  Analysis  technique, 
a  new  perspective  of  the  capabilities  and  limitations  of  the  enemy  can  be  obtained),  and 
development  of  tactical  units  ,  procedures,  and  systems  (particularly  within  the 
command  and  control  domain).  We  also  see  possibilities  to  develop  the  Combat 
Efficiency  Analysis  technique  into  a  generic  mission  evaluation  tool  for  activities  other 
than  those  of  warfare,  e.g.  crisis  management  and  emergency  response.  However,  the 
potential  risk  of  a  workload  increase  has  not  yet  been  fully  investigated.  If  workload 
actually  increases  significantly  in  a  way  that  would  endanger  the  outcome  of  an  actual 
combat  mission,  this  must  be  taken  into  consideration  before  launching  a  within-mission 
Combat  Efficiency  Analysis.  If  this  should  be  the  fact,  the  Combat  Efficiency  Analysis 
technique  will  perform  very  well  in  a  post-mission  debriefing  situation  as  well,  judging 
from  the  successful  Army  Combat  Training  Center  achievements. 
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Abstract 

Twenty  subjects  drove  a  normal  car  and  an  Automated  Highway  System  (AHS)-car  in  a 
high  fidelity  driving  simulator.  The  AHS -vehicle  was  driven  in  three  conditions;  at  1 
second  time  headway,  at  0.25  s.  time  headway  and  as  platoon  leader.  In  the  final 
condition  when  they  were  driving  as  platoon  leader,  the  front  sensor  of  the  automated 
vehicle  failed  to  detect  a  merging  car  in  time,  leading  to  a  critical  incident  (but  not  an 
accident).  Ratings  of  mental  effort,  activation,  risk,  safety  and  acceptance  were  collected 
and  the  driver’s  ECG  was  continuously  registered. 

Results  show  that  normal  driving  is  more  activating  and  effortful  than  automated  driving. 
When  driving  in  AHS  mode  drivers  prefer  a  large  time-headway  to  vehicles-in-front. 
Risk  ratings  are  highest  for  the  emergency  situation,  followed  by  driving  at  0.25  seconds 
in  automated  mode.  The  emergency  situation  was  surprising  for  most  subjects,  and  the 
majority  did  not  respond  or  responded  late  to  it  (drivers  could  avoid  the  critical  situation 
by  pressing  the  brake). 

Introduction 

In  the  Automated  Highway  System  (AHS,  e.g.,  Sussman,  1996)  “intelligenf’cars  will  be 
guided  by  the  road  infrastmcture  and  will  control  their  own  lateral  and  longitudinal 
position  on  the  road.  Traffic  on  an  automated  lane  will  drive  in  platoons,  i.e.  in  rows  of 
cars  driving  at  short  headway.  The  ultimate  aim  of  AHS  is  to  increase  road  capacity  and 
to  eliminate  driver  error  (Congress,  1994).  However,  for  at  least  several  years  after  initial 
AHS  implementation.  Safety  Boards  will  require  that  the  driver  ‘stays  in  the  loop’.  The 
driver  will  be  strongly  encouraged  or  perhaps  even  legally  requested  to  monitor  the  AHS 
and  other  vehicle  systems  during  the  journey,  to  ensure  that  all  functions  are  working 
properly  (Alicandri  &  Moyer,  1992).  For  reasons  of  liability  the  user  has  to  stay  alert  in 
the  AHS  facility  aU  the  time,  just  in  case  manual  control  needs  to  be  resumed  (Congress, 
1994),  This  is  similar  to  an  aeroplane  pilot  who,  monitoring  autopilot  functioning, 
resumes  control  whenever  deemed  necessaiy.  However,  driving  a  car  entails  higher 
frequencies  of  actions  and  lower  time  constants,  i.e.  it  will  be  much  more  difficult  to  take 
over  car  control  than  aeroplane  control  in  case  of  acute  emergency. 

Only  after  system  reliability  is  ensured  and  driver  liability  is  ruled  out,  driver  attention  to 
the  system  will  become  formally  superfluous.  One  of  the  basic  questions  then  in  AHS,  in 
due  course,  will  be  whether  driver  intervention  with  vehicle  and  system  operation  in  an 
emergency  situation  should  be  allowed  or  not  (Tsao  et  al.,  1993).  If  the  driver  is  expected 
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to  intervene,  training  will  be  required.  If  s/he  is  not  supposed  to  intervene,  one  of  the 
questions  raised  then  is  whether  s/he  will  try  to  do  so  anyway  (Tsao  et  al.,  1993),  Finally, 
will  it  be  acceptable  to  the  driver  to  be  totally  without  control.  The  question  whether 
drivers  are  going  to  give  over  control  is  the  question  whether  AHS  is  accepted.  Research 
indicates  that  drivers  prefer  warnings  to  actual  take-over  of  control  (Nilsson  &  Aim, 
1991,  Hoedemaeker,  1996),  with  the  exception  of  driver  emergency  situations  (e.g.,  in 
case  of  an  acute  health  problem  or  when  falling  asleep,  Petica  &  Bekiaris,  1996).  Shifting 
the  driver  ‘out-of-the-loop’  may  lead  to  reduced  responsiveness  to  critical  events,  which 
has  been  reported  as  pre-crash  factor  in  aviation  (e.g.,  Ward,  1996).  Automation  may  also 
increase  reaction  time.  In  case  of  continuous  monitoring,  reaction  time  to  events  in  a 
driving  task  can  be  restricted  to  something  like  one  second,  while  if  more  than  one 
functions  have  to  be  monitored  and  other  tasks  are  attended  to,  awareness  of  the  situation 
has  to  be  refreshed  with  increased  frequency  and  the  malfunction  and  its  origin  have  to  be 
determined  which  might  take  many  seconds.  In  this  way,  an  attempt  to  reduce  workload 
is  actually  very  likely  to  lead  to  increased  workload  (Hancock  &  Parasuraman,  1992), 
The  classic  goal  of  automation  is  to  replace  human  manual  control,  planning  and  problem 
solving  by  automatic  devices.  However,  these  systems  still  need  human  beings  for 
supervision  and  adjustment.  It  has  been  suggested  that  the  more  advanced  a  control 
system,  the  more  crucial  is  the  contribution  of  the  human  operator  (Bainbridge,  1983). 

The  point  made  by  Bainbridge  (1983)  is  as  follows:  normal  operation  is  performed 
automatically,  abnormal  conditions  are  to  be  dealt  with  manually.  Unfortunately,  as  a 
result  of  automation,  experience  is  limited,  while  in  case  of  abnormal  conditions  (i.e., 
something  is  wrong  with  the  process)  unusual  actions  will  be  required.  Also,  human 
problem  solving  is  not  optimal  under  time-pressure.  Monitoring  of  (present)  automatic 
processes  is  based  on  skills  that  formerly  manual  operators  have,  and  that  future 
generations  of  operators  (/drivers)  cannot  be  expected  to  have  (Bainbridge,  1983).  Pilots 
also  indicated  that  although  automation  reduced  workload,  it  also  had  a  negative  effect  on 
flying  skills.  They  considered  manually  flying  of  a  part  of  every  trip  important  to 
maintain  these  skills  (McClupha  et  al.,  1991). 

The  aim  is  free  of  faults  functioning,  but  it  is  obvious  that  there  is  no  way  of  guaranteeing 
that  a  car  or  any  other  part  of  the  system  will  not  fail  in  the  AHS  (Hitchcock,  1991). 
When  a  system  fails  to  work  or  is  in  a  state  that  failure  is  possible,  feedback  should  be 
provided  in  order  to  let  the  driver  know  that  s/he  can  not  rely  on  the  system.  The  main 
reason  for  this  is  that  automated  systems  can  and  will  lead  to  what  has  been  called 
‘complacency’  (Wiener  and  Curry,  1980).  Complacency  is  an  attitude  of  (over)reliance 
on  an  automated  system.  This  and  other  forms  of  behavioural  adaptation  or  compensation 
as  it  is  called  in  a  wider  field,  are  factors  that  should  be  taken  into  account  when 
investigating  the  conditions  for  introduction  of  AHS  (Verwey,  Brookhuis  &  Janssen, 
1996). 

The  new  AHS-driving  task  will  bring  the  driver  in  an  unknown  situation,  for  the  which 
the  demands,  limits  and  preferences  are  not  yet  determined.  Questions  of  major 
importance  in  this  respect  are: 

•  What  is  the  driver’s  opinion  about  this  altered  task,  what  is  the  level  of  acceptance? 

•  Is  there  an  effect  of  driver  alertness  level  on  task  performance  and  acceptance? 
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•  Are  there  any  differences  in  driver  behaviour  and  risk  ratings  for  different  following 
distances  in  a  platoon  of  cars? 

•  Are  there  any  differences  in  driver  behaviour  and  risk  ratings  when  driving  as 
platoon-leader  compared  with  driving  inside  the  platoon  of  cars? 

•  What  happens  in  case  of  system  failure  in  an  emergency  situation,  will  the  driver  (try 
to)  take  over  control? 

Bringing  the  driver  into  the  real  AHS  condition  is  not  yet  possible,  reason  to  investigate 
driver’s  behavioural  changes  and  feelings  in  a  driving  simulator.  Another  reason  to  carry 
out  AHS  studies  in  a  driving  simulator  is  an  ethical  one,  any  emergency,  any  system  or 
driver  failure  must  be  without  any  (physical  and/or  material)  consequences. 


Method 

Twenty  subjects  drove  the  advanced  driving  simulator  of  the  Centre  for  Environmental 
and  Traffic  Psychology  (the  former  Traffic  Research  Centre,  Van  Wolffelaar  &  Van 
Winsum,  1995).  All  subjects  drove  the  simulator  in  the  automated  mode  where  lateral  and 
longitudinal  position  was  regulated  by  the  system,  as  well  as  in  the  ‘normal’  mode  (as  if 
in  an  ordinary  hand- shifted  car).  Subjects  completed  two  sessions  in  the  simulator,  one  in 
the  morning  (between  9  an  12  AM)  and  one  in  the  evening  (between  9  and  1 1  PM),  hi  the 
evening,  subjective  driver  alertness  can  be  expected  to  be  lower  (Monk  et  al.,  1983,  De 
Waard  et  al.,  1997)  which  could  be  a  nonoptimal  state  for  monitoring.  The  order  of  the 
conditions  was  balanced  across  subjects. 

In  the  AHS  conditions  drivers  drove  (or  actually  only  sat  in)  a  car  in  a  platoon.  The  car 
drove  at  high  speed.  Three  conditions  were  part  of  the  experiment:  driving  in  the  platoon 
at  1  second  time  headway  to  the  car-in-front,  driving  at  0.25  seconds  headway,  and 
driving  as  platoon  leader.  The  experimental  following  times  that  were  used  have  been 
proposed  in  the  USA  by  Bloomfield  (1995).  The  driver  was  not  able  to  control  lateral 
position,  i.e.,  steering  is  not  possible.  The  only  way  to  overrule  the  AHS  was  by  applying 
the  brake. 

During  the  rides  physiological  measurements  (ECG)  and  self-report  ratings  were 
collected.  These  measurements  provide  information  on  mental  effort,  activation  and 
experienced  risk  and  safety.  Failure  of  the  AHS  can  and  was  tested  only  once,  in  the  final 
session  at  the  end  of  the  experiment  when  they  were  driving  as  platoon  leader,  in  order  to 
prevent  negative  effects  on  the  rest  of  the  experiment  and  the  simulator  in  general.  The 
emergency  was  a  failure  of  the  AHS  front-sensor  to  detect  a  merging  car.  The  car  merged 
extremely  close  in  front  of  the  AHS  vehicle  (leaving  0. 1  metres  of  space  between  the 
cars).  Failure  of  the  AHS  did  not  lead  to  an  accident,  but  the  situation  was  such  that  in 
normal  driving  an  alert  subject  would  apply  the  brake. 

Before  the  actual  experiment,  subjects  were  asked  to  read  a  description  of  the  AHS  and 
were  trained  in  driving  the  simulator,  both  in  normal  and  in  AHS  mode.  The  order  of 
these  conditions  was  balanced  across  subjects.  Per  visit,  subjects  drove  under  the 
following  conditions: 
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N:  Normal,  conventional  driving,  distance  to  a  lead  car  is  self-chosen 

Al:  AHS,  supervisory,  distance  to  car-in-front  =  1  second,  speed  =  100  km/h 

A.25:  AHS,  supervisory,  distance  to  car-in-front  =  0.25  seconds,  speed  =100  km/h 
F:  AHS  platoon  leader,  supervisory.  This  condition  was  only  part  of  the  experiment 

during  the  first  session  (visit). 

F*:  AHS  failure  to  respond.  In  this  condition  the  subject’s  car  was  platoon  leader,  and 

a  car  merged  from  the  emergency  lane  to  subject’s  path.  This  condition  was  only 
part  of  the  experiment  during  the  second  session. 


In  the  middle  part  of  the  N,  Al  and  A.25  conditions  the  lead  car  slowed  down,  in  the 
AHS  conditions  the  system  reacted  appropriately  (i.e.,  also  slowed  down).  The  order  of 
conditions  was  balanced  over  subjects.  The  following  measures  were  sampled  at  5  Hz: 
lateral  position,  steering  wheel  position,  time  headway,  speed  (aU  for  conventional 
driving)  and  brake  pedal  position.  Changes  in  driver  alertness  are  preferably  measured 
from  alertness  related  physiological  measures  (see  Brookhuis  &  De  Waard,  1993).  As 
physiological  measure  subject’s  heart  rate  was  registered.  From  these  measurements 
average  heart  rate  and  heart  rate  variability,  in  particular  in  the  frequency  domain,  were 
calculated.  Variability  in  the  Blood  pressure  band  (0.07-0.14  Hz,  “the  -0.10  Hz 
component”)  reflects  mental  effort  (see  e.g.,  Mulder,  1992,  Brookhuis  &  De  Waard, 
1993). 

Before  each  session  subjects  completed  questionnaires  on  sleepiness  (Stanford  Sleepiness 
Scale  and  a  rating  of  tiredness,  see  De  Waard,  Van  der  Hulst,  &  Brookhuis,  1997).  Before 
the  first  test  ride  on  the  first  day  and  after  the  last  ride  on  the  second  day  acceptance  of 
the  AHS  was  assessed  by  a  standard  ATT  acceptance  checklist  (Van  der  Laan,  Heino  & 
De  Waard,  1997).  After  each  condition  (i.e.,  after  Al,  A.25,  N  and  F)  a  self-rating  of 
mental  effort  (Rating  Scale  Mental  Effort,  RSME,  Zijlstra,  1993),  a  rating  of  activation 
(Bartenwerfer,  1969),  a  risk  rating  (Heino,  1996)  and  a  safety  rating  on  a  5-point  Likert 
scale  were  collected.  Subject’s  ECG  while  resting  was  measured  at  the  start  and  end  of 
each  session. 

At  the  end  of  the  second  session  subjects  again  completed  the  acceptance  checklist  under 
the  instruction  not  to  take  into  account  the  condition  where  the  system  failed.  A  preferred 
AHS-time  headway  was  also  indicated. 

Results 

Average  age  of  the  subjects  was  29.8  {sd  6.0),  20  %  was  female.  Their  average  mileage 
was  16  (XX)  km/year  and,  on  average,  they  had  held  a  licence  for  8.6  years  (sd  5.7). 
Effects  of  time-of-day  were  limited  to  subjective  tiredness,  which  was  higher  in  the 
evening.  As  other  effects  of  time-of  testing  (and  subjective  alertness  level)  were  absend, 
no  further  attention  will  be  given  to  this  factor. 

Average  speed  in  the  ‘normal-driving  condition’  was  equal  to  the  lead  car,  100  km/h. 
Average  time-headway  was  2.2  seconds  (sd  1.4),  well  above  the  AHS  time  headways  of  1 
and  0.25  seconds. 

What  happened  in  the  emergency  situation  in  the  final  trial  is  illustrated  in.  At  t=0  the 
experimental  AHS  vehicle  (E)  is  within  viewing  distance  of  the  car  that  will  merge  (M), 
which  is  still  standing  still  on  the  emergency  lane.  At  t=7  seconds,  the  M-car  accelerates. 


178 


while  at  t=9.7  s  the  M-car  blinks  and  starts  to  merge  in  front  of  E.  At  t=14.7  s  the 
manoeuvre  is  completed,  and  if  the  subject  did  not  brake,  the  distance  between  the  two 
vehicles  would  be  no  more  than  0. 1  metres. 


IMU — 

Emergency  lane 


t=  0 


t=7.0  t=9.7  t=14.7 


Figure  1.  Emergency  situation  (F*) 


In  Table  1  subject’s  behaviour  during  this  trial  is  classified  on  the  basis  of  braking- 
response,  the  only  means  to  interfere.  As  much  as  50%  of  the  subjects  did  not  react  at  all, 
whereas  15%  braked  after  the  car  had  merged  close  in  front. 


Table  L  Subject’s  response  during  the  emergency  situation 


Response 

-  -’O . .  <3 

time  (t  =) 

Proportion  subjects 

No  reaction 

oo 

50% 

Braked 

late 

>  14  s 

15% 

Braked 

middle 

9-14  s 

30% 

Braked 

early 

<  9  s 

5% 

After  each  conditions  self-ratings  of  mental  effort,  activation,  risk  and  safety  were 
collected.  In  Table  2  averages  of  these  ratings  are  summarised.  Statistical  tests  (the  reader 
is  referred  to  the  technical  report,  De  Waard  et  al.,  1997)  indicate  that  normal  driving  is 
rated  as  more  effortful  and  more  activating.  The  short-headway  AHS  condition  requires 
more  mental  effort,  is  more  activating  and  riskier,  and  judged  to  be  less  safe.  The 
emergency  situation  is  considered  to  be  the  riskiest  and  the  least  safe.  There  were  no 
significant  effects  of  being  inside  or  in  front-position  of  the  platoon  on  any  of  the 
subjective  ratings. 
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Table  2:  Averages  of  self-ratings.  The  following  ranges  apply,  from  low(effort,  risk  etc)  to  high:  Effort 
(RSME):  0-150,  Activation  0-270,  Risk  0-6  (7  point).  Safety  1-5  {5-point).  Session  1  is  the  first  visit  and 
test.  Session  2  the  second  visit,  thus  on  another  day. _ _ _ _ _ 


Condition 

Session 

Effort 

Activation 

Risk 

Safety 

Normal 

1 

41 

136 

2.0 

1.6 

2 

32 

119 

1.5 

1.6 

AHS  1  s 

1 

12 

85 

1.1 

1.4 

2 

13 

79 

1.0 

1.4 

AHS  0,25  s 

1 

21 

101 

2.2 

2.3 

2 

18 

96 

1.7 

2,0 

Lead  (F) 

I 

16 

80 

1,1 

1.5 

Emergency 
(Lead  F*) 

2 

28 

110 

2.9 

2.9 

If  the  AHS  conditions  are  taken  together  and  are  compared  to  normal  driving,  a 
significant  effect  on  the  0.10  Hz  component  of  heart  rate  variability  is  found.  During 
normal  driving  the  0.10  Hz  component  is  more  supressed  than  during  automated  driving, 
which  indicates  increased  mental  effort  during  normal  driving.  All  heart  rate  parameters 
differ  significantly  from  rest  during  driving.  More  details  on  heart  rate  can  be  found  in 
the  technical  report  (De  Waard  et  al.,  1997). 

Driver  acceptance  of  AHS  was  measured  using  standard  ATT  acceptance  items  (Van  der 
Laan,  Heino  &  De  Waard,  1997).  Before  the  actual  tests,  drivers  were  asked  to  judge  the 
AHS  system,  and  after  the  second  session  the  same  was  done  with  the  explicit  instmction 
not  to  take  into  account  the  emergency  trial.  Reason  for  this  is  the  interest  in  general  AHS 
acceptance,  and  the  unlikeliness  of  acceptance  of  an  ill-functioning  system.  Usefulness  of 
the  AHS  is  slightly  positive  (0,5  on  a  scale  from  -2  to  -i-2),  in  terms  of  being  satisfying 
the  overall  opinion  is  slightly  negative  (-0,3),  Acceptance  ratings  did  not  change 
significantly  after  experience  with  the  system  and  are  comparable  to  the  evaluation  of  an 
autonomous  intelligent  cruise  control  (see  Van  der  Laan,  Heino  &  De  Waard,  1997),  i.e. 
relatively  low  scores  due  to  take-over  of  control. 

75%  of  the  subjects  preferred  the  long  (1  s.  time  headway)  following  condition  to  the 
short  (0.25  s.  time  headway)  following  condition.  The  other  25%  did  not  prefer  one 
condition  to  the  other. 


In  addition  to  driver  acceptance  in  terms  of  usefulness  and  satisfaction,  questions  were 
asked  whether  the  driver  agreed  with  various  positive  and  negative  aspects  of  the 
systems,  which  were  explained  in  one  or  two  sentences.  Table  3  shows  the  percentages  of 
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the  drivers  who  agree  on  four  of  the  statements  about  the  Automated  Highway  System 
(AHS)  before  and  after  the  tests. 

Table  3.  Percentages  of  AHS  opinions. 


Before 

After 

positive: 

It  enhances  traffic  flow 

80% 

65% 

positive: 

I  don’t  have  to  be  attentive 

45% 

45% 

negative: 

No  control  of  driving 

55% 

75% 

negative: 

No  fun  of  driving 

75% 

75% 

The  Wilcoxon  Signed  Ranks  test  shows  that  the  increase  in  ‘No  control  of  driving’  is 
significant.  Much  more  drivers  disliked  the  fact  that  they  had  no  control  of  driving  when 
driving  in  the  AHS  system  after  experience  with  the  system.  On  the  other  three  items 
drivers  did  not  change  their  opinion  after  experience  with  the  system,  or  in  the  direction 
of  a  less  positive  opinion. 


Conclusions 

In  the  present  experiment  differences  between  automated  driving  and  normal  driving 
were  found  on  mental  effort  and  activation.  More  effort  is  required  for  normal  driving,  as 
is  indicated  by  self-reports  of  invested  effort  and  heart  rate  parameters.  Driver  activation 
is  also  higher  during  normal  driving  compared  to  automated  driving.  The  results  on  these 
self-report  scales  are  in  line  with  expectations  with  respect  to  an  altered  driving  task; 
during  conventional  driving  the  driver  has  to  be  active,  during  automated  driving  the  task 
is  supervisory.  Risk  and  safety  ratings  differ  to  a  lower  extent  and  depend  upon  the  time¬ 
headway  of  the  AHS.  At  the  close  headway  of  0.25  s,  the  average  risk  rating  is 
comparable  to  normal  driving  whereas  safety  during  the  automated  drive  is  judged  to  be 
lower.  At  the  larger  headway  of  1  second  these  differences  disappear,  and  risk  is  actually 
rated  lower  during  the  automated  ride  compared  with  the  normal  ride.  When  asked  to 
indicate  a  preferred  AHS  following  distance  the  majority  chooses  the  large  headway, 
while  none  of  the  subjects  prefers  the  short  headway  (25%  does  not  have  a  preference). 
One  of  the  advantages  of  AHS  is  an  increased  road  capacity  per  driving  lane.  This  effect 
is  enhanced  by  shorter  time-headways,  but,  driver  acceptance  of  this  is  a  prerequisite.  In 
addition  to  the  preference  as  indicated  directly  by  the  subjects,  the  higher  risk  ratings  for 
the  short  following  conditions  also  give  a  clue  that  introduction  of  short  time  headways  in 
platoon  driving  may  be  stopped  by  low  user  acceptance.  Similar  effects  were  found  in 
studies  by  De  Vos  (De  Vos  et  al,  1996,  De  Vos  &  Hoekstra,  1997),  drivers  dislike  short 
time  headway,  De  Vos  and  coworkers  expected  increased  ratings  of  comfort  when 
driving  at  very  close  distances  from  the  vehicle-in-front,  but  did  not  find  them.  Their 
hypothesis  was  that  even  though  at  short  headways  the  chance  of  collision  may  be  high, 
the  collision  energy  (and  thus  damage)  is  low  due  to  small  speed  differences  when 
colliding.  Perhaps  for  most  people  this  positive  implication  is  difficult  to  imagine. 


The  emergency  situation  at  the  end  of  the  experiment  was  a  surprise  for  most,  as  many 
did  not  react  or  responded  fairly  late.  In  most  instances  where  the  front-sensor  failed  to 
work,  the  tested  situation  would  have  led  to  a  collision.  Complacency  may  be  the  reason 
for  no  response,  after  the  experiment  a  subject  stated  that  “you  either  trust  the  system  and 
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sit  and  relax  or  don’t  trust  the  system  and  are  continuously  alert  and  stressed”.  One 
should  be  aware  of  the  fact  that  subjects  were  tested  in  a  driving  simulator,  which  may 
have  had  an  effect  on  their  behaviour.  The  effects  on  motivation  of  driving  in  a  simulator 
opposed  to  on-the-road  tests  have  been  described  in  the  literature  (e.g.,  Smiley  & 
Brookhuis,  1987).  The  actual  risk  of  collision  in  the  present  experiment  was  nil  and  may 
have  made  subjects  less  inclined  to  take  over  control. 

Driver  opinion  on  different  Automated  Vehicle  Guidance  systems  indicates  that  drivers 
expect  the  systems  to  have  as  much  negative  as  positive  effects  when  introduced  into  the 
traffic  system.  In  general  it  can  be  concluded  that  drivers  are  positive  about  the  effects  on 
traffic  safety,  but  they  see  also  negative  effects  like  loosing  control  of  driving  and 
attention  reduction.  This  result  is  in  accordance  with  the  results  of  an  earlier  study  with 
this  questionnaire  (Hoedemaeker,  1996).  Driver  acceptance  of  a  well-functioning  AHS  is 
neutral  in  terms  of  satisfying  and  usefulness.  This  means  that  such  a  system  is  not  refused 
after  experience  in  a  simulator.  It  is,  however,  obvious  that  an  AHS  should  be  fail-safe 
and  that  drivers  should  be  able  to  trust  on  accurate  functioning  of  the  system. 
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Summary 

When  assessing  novel  car  technologies  like  driver  assistance  systems  or  route  guidance 
systems  it  is  desirable  to  predict  the  effects  of  these  technologies  on  vehicle-driver- 
environment  interaction  already  during  the  development  phase.  Due  to  the  complexity  of 
the  issue  computer  simulation  is  the  most  appropriate  means  for  such  an  evaluation.  As 
human  factors  play  an  important  role  in  the  performance  of  the  entire  system,  the  need  of 
reliable  conclusions  implies  an  appropriate  modelling  of  human  decision  making  and 
driving  behaviour. 

In  this  paper  a  model  of  individually  acting  drivers  which  have  cognitive  capabilities  like 
planning,  decision  making  and  goal  directed  acting  is  outlined.  The  model  presented  is 
based  on  a  production  system,  which  can  be  extended  by  the  instrumentality  of  Artificial 
Neural  Networks  and  Fuzzy  Techniques  to  describe  psychological  knowledge  about 
motivation,  information  processing  capabilities  and  motor  control.  The  behaviour  of  one 
or  more  drivers  in  typical  traffic  situations  is  simulated.  This  includes  also  the  modelling 
of  inter  social  relations. 

The  driver  model  is  implemented  and  run  in  a  driving  simulation  environment.  Some 
selected  examples  are  presented  and  discussed. 

Introduction 

Traffic  accidents  are  frequently  caused  by  perceptual  flaws  or  mutual  misunderstandings 
of  drivers  in  quickly  emerging  critical  situations,  especially  at  high  speeds.  Microscopic 
traffic  simulation  that  focuses  on  individual  behaviour  of  the  human  as  well  as  the 
driver-vehicle  interaction  could  help  to  gain  more  insight  into  the  origination  of 
accidents  and  the  means  how  to  prevent  them  as  well  as  predict  the  effects  of  driver 
assistance  systems  and  influence  their  development  in  an  early  stage.  We  therefore 
concentrate  on  modeling  and  simulating  of  human  cognitive  capabilities  and 
performance  in  a  typical  dynamic  situation  considering  social  interaction  with  other  road 
users. 

Driver  behaviour  is  determined  in  a  decisive  way  by  accessible  information  about  the 
current  situation,  experience  and  skill  as  well  as  motivation.  The  necessary  information 
about  the  environment  is  gathered  by  means  of  the  human  sensory  apparatus  which 
cannot  reproduce  quantities  like  relative  velocities  or  distances  with  great  accuracy. 
Experience  and  skill  in  car  driving  manifests  in  the  capability  to  judge  the  significance 
of  available  information  about  the  actual  driving  situation  and  the  existence  of  highly 
trained  complex  motor  programs  like  parking  or  overtaking.  Together  with  motivation 
which  is  a  central  factor  in  the  entire  cognitive  process,  these  factors  influence  the 
internal  setting  of  goals,  judgment  of  a  situation,  the  decision  process  and  the  way  of 
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executing  actions.  Goal  directed  acting  in  an  environment  populated  by  other  agents 
implies  moreover  the  evaluation  of  the  consequences  of  one’s  decisions  and  actions  and 
hence  anticipation  of  the  behaviour  of  other  traffic  participants. 

We  conclude  that  a  driver  model  for  microscopic  traffic  simulation  which  can  be  used  to 
simulate  individual  behaviour  in  critical  situations  has  to  embody  human  perception 
characteristics  and  motor  processes  as  well  as  motivation,  decision  making  and  goal 
directed  acting. 

Other  Driver  Models 

To  simulate  driver  behaviour  it  is  most  common  and  often  sufficient  to  model  the  driver 
by  means  of  control  theory  where  driver  behaviour  is  described  by  a  single,  basically 
linear  differential  equation  with  time  shifting  operations  and  possibly  a  nonlinear  part. 
The  individual  attributes  of  a  driver  are  encoded  in  the  structure  of  the  differential 
equation  and  its  parameters  that  can  be  adjusted  such  that  „carefur‘  or  „daring  driver 
behaviour  can  be  simulated  in  a  given  situation  (Wiedemann,  1974).  Intrinsically  human 
characteristics  like  goal  directed  action  that  is  not  a  mere  stimulus  reaction,  but  a  result 
of  planning  and  decision  making  influenced  by  particular  motivational  factors  cannot  be 
modelled  that  way,  though.  To  achieve  the  objective  to  model  individual  behaviour, 
psychological  and  physiological  knowledge  about  human  cognition  has  to  be 
incorporated  into  the  model. 

In  cognitive  psychology  production  systems  in  which  knowledge  and  behaviour  are 
represented  in  the  form  of  IF  -  THEN  rules  are  commonly  used  to  model  task 
performance  and  knowledge  representation.  Proved  program  systems  like  Soar  und  ACT- 
R  (Newell,  1990;  Anderson,  1993)  which  are  based  on  psychologically  motivated 
problem  solving  procedures  and  algorithms  have  already  been  used  for  the 
implementation  of  cognitive  driver  models  (Aasman,  1995).  As  the  process  of  problem 
solving  which  is  equivalent  to  the  decision  process  in  our  case  is  part  of  the  program 
design  of  the  above  mentioned  program  systems,  we  believe  that  a  specially  designed 
program  better  serves  the  purpose  of  getting  some  insight  particularly  into  the  relevance 
of  motivation  and  social  interaction  on  decision  making. 

Another  approach  of  to  model  driver  behaviour  is  that  of  hybrid  modelling  (JUrgensohn, 
1997).  The  idea  is  to  combine  different  modelling  tools  like  Fuzzy  Techniques,  Artificial 
Neural  Networks,  production  systems  or  control  theory  in  order  to  take  advantage  of  the 
particular  qualities  each  method  can  contribute  to  the  modelling  of  human  behaviour 
(Wolter  et  al„  1997;  JUrgensohn  &  Willumeit  1997).  A  comprehensive  survey  of 
existing  driver  models  was  presented  by  (Willumeit  &  JUrgensohn,  1996). 

Description  of  the  driver  model 

Individual  behaviour  is  defined  through  motivational  factors,  overall  goals,  specific 
conflict  resolution  strategies,  estimation  qualities,  driving  skills  and  given  declarative 
and  procedural  knowledge  about  traffic  rules,  vehicle  operation  as  well  as  driver  specific 
motor  programs. 

Perception,  decision  making  and  the  execution  of  movements  or  motor  programs  all 
involve  processing  times  which  can  play  an  important  role  especially  in  dynamic 
situations  at  high  speeds. 
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The  program  is  structured  in  the  sense  of  object  oriented  design  by  means  of  data 
structures  and  corresponding  methods.  Driver  behaviour  is  then  described  by  element 
functions  which  can  contain  rules  like  production  systems  but  also  parametric  functions 
and  programs.  This  conception  can  be  found  especially  for  multiagent  systems  (Huhn, 
1991). 

In  figure  1  the  realized  model  structure  is  sketched  in  principle.  It  should  be  noted  that 
the  separate  blocks  with  the  denoted  functionalities  cannot  be  interpreted  in  a  way  that 
these  functions  are  independent  from  each  other,  although  from  the  viewpoint  of 
computer  implementation  it  makes  sense  to  treat  them  as  seperate  modules. 


external  world 


Figure  1 :  Model  Structure  of  the  Cognitive  Driver  Model 


Coordination  of  Subtasks 

The  driving  task  in  an  environment  populated  by  a  number  of  other  agents  can  be 
considered  to  be  composed  of  several  subtasks  which  have  to  be  to  be  handled 
simultaneously  as  well  as  consecutively  during  driving.  Some  obvious  subtasks  of  car 
driving  which  are  relevant  for  the  present  investigation  are  listed  below  : 

Sensation  of  environment  (expressed  by  state  variables  ) 

Identification  and  interpretation  of  traffic  situations 

Decision  among  several  alternatives  of  action  through  evaluating  of  resulting 
consequences 

Motoric  execution  of  an  action 


Regulation  of  the  action  after  comparison  with  estimated  outcome 
Communication  with  other  traffic  participants 


Coordination  of  subtasks 
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For  the  coordination  of  the  subtasks  some  assumptions  about  human  resources  in 
information  processing  and  motoric  executions  are  necessary.  It  is  presupposed  that 
perception,  decision  making  and  motoric  action  can  be  executed  simultaneously, 
bottlenecks  occur  only  when  similar  tasks  are  to  be  performed.  Therefore  the  main 
criterion  for  subtask  coordination  is  chronological  order  and  consideration  of  time 
constants  for  the  respective  execution.  This  simple  approach  is  certainly  not  valid  for  a 
comprehensive  cognitive  modelling,  but  simulation  results  of  multiple-task  performance 
in  the  field  of  human-computer  interaction  assumptions  match  well  with  observed 
behaviour  (Kieras  &  Meyer,  1992). 

Perception 

Drivers  have  only  limited  access  to  important  state  variables  of  the  environment  stored  in 
the  public  database,  limits  being  due  to  human  sensory  characteristics.  The  state 
variables  listed  below  are  presumed  to  describe  the  driver’s  world  and  therefore  are  made 
available  to  the  model: 

•  Width  and  course  of  the  road 

•  own  velocity  and  position 

•  type  of  neighbouring  objects 

•  distance,  relative  velocity  and  acceleration  to  neighbouring  objects 

•  steering  movements  and  movement  characteristics  (smoothness)  of 
neighbouring  objects 

•  Braking,  blinking,  flash  of  neighbouring  objects 

•  traffic  rules 

•  additional  information  from  guidance  or  driver  assistance  systems 


In  the  model  important  characteristics  of  human  sensation  should  be  reflected.  This  was 
realized  by  processing  the  quantities  read  from  the  database  in  a  particular  way  to 
account  for  the  following  features: 

•  limitation  to  visual  information  gathering  only 

•  limited  visual  field  (objects  are  eventually  not  visible) 

•  erroneous  estimation  of  relative  distances,  velocities  and  accelerations 

•  finite  processing  time  for  object  perception 

•  objects  perceived  peripherally  imply  information  insufficiency  and  further 
delay  in  perception  due  to  required  eye  movements 


Errors  in  estimating  distances  etc.  are  affected  by  the  driver’s  expertise  and  by 
motivational  factors.  The  resulting  vagueness  of  perceived  values  is  modelled  by  adding 
noise  to  the  database  quantitites. 
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Reported  processing  times  for  visual  perception  vary  between  50  and  250  ms.  These 
differences  reflect  specific  processing  speeds  for  different  visual  resolution  (peripheral  vs 
foveal)  on  the  one  hand  and  the  influence  of  the  test  environment  on  the  other  hand. 

Identification  of  Situation 

For  the  driver  model  the  state  of  the  world  is  represented  by  the  aggregation  of  a  limited 
number  of  state  variables  relevant  for  the  driving  task  like  position  or  velocity,  which  are 
continuously  changing  with  time.  In  cognitive  psychology  the  term  „situation“  is 
understood  as  a  kind  of  category  which  encompasses  a  certain  range  within  the  state 
space  and  is  closely  related  with  a  corresponding  action.  This  is  modelled  in  such  a  way 
that  the  identification  of  the  situation  is  performed  by  a  classification  of  the  current 
variables  of  state.  Classification  here  means  that  a  continuous  state  vector  is  mapped 
onto  a  number  of  discrete  situational  categories. 

The  identification  of  a  situation  can  also  be  considered  as  a  kind  of  pattern  recognition,  a 
typical  field  of  application  for  Artificial  Neural  Networks  (ANNs).  Previous  work  shows 
that  ANNs  are  well  suited  to  model  humanlike  identification  of  a  situation  with 
subsequent  estimation  of  action  urgency  (Jiirgensohn  et  al.,  1994;  Jurgensohn  & 
Willumeit,  1997;  Jurgensohn  1997). 

We  chose  to  classify  a  situation  by  providing  the  driver  model  with  declarative 
knowledge  about  traffic  rules  and  individual  safety  margins  that  are  influenced  by 
motivational  factors.  The  model  compares  the  perceived  variables  of  state  with  its  rule 
base  and  the  identified  situation  with  an  internally  desired  state.  If  the  deviations  are 
above  a  certain  margin,  a  drive  to  change  the  current  situation  a  specific  way  to  approach 
the  desired  state  is  created.  As  there  may  be  several  action  alternatives  that  are  able  to 
transfer  the  present  state  into  a  more  favourable  one  a  decision  process  becomes 
necessary  to  evaluate  the  alternatives  and  make  a  choice. 

Decision  making 

Concurring  action  alternatives  require  a  decision.  In  order  to  evaluate  distinctive  actions 
the  action  sequences  have  to  be  planned  first,  in  a  next  step  consequences  due  to 
reactions  of  other  traffic  participants  have  to  be  taken  into  account  and  evaluated  with 
respect  to  own  goals  and  security.  Goals  are  evaluated  through  estimation  of  benefit  and 
cost  (risk).  For  this  purpose  the  individual  needs  some  knowledge  about  implications  of 
other  traffic  participants’  behaviour  and  signals  they  possible  convey  to  him. 
Additionally  the  model  will  have  to  make  some  assumptions  about  the  reactions  of  other 
traffic  participants  on  own  behaviour,  which  are  deduced  from  observing  the  driving 
style  of  other  drivers  or  from  their  communications. 

The  presented  model  has  only  two  action  alternatives  to  choose  from:  following  and 
overtaking.  The  decision  will  be  largely  influenced  by  motivation  factors,  that  is  how  the 
benefit  of  overtaking  compares  with  the  risk  of  an  accident  due  to  the  driver  approaching 
from  behind.  The  risk  estimation  itself  will  also  depend  on  motivation  and  driving  skills 
of  the  model. 

If  a  decision  has  been  made,  the  situation  changes  and  thus  the  possible  actions  the 
driver  can  perform.  In  the  state  of  following  there  are  the  possibilities  to  approach  or 
brake,  in  the  state  of  overtaking  there  are  the  possibilities  of  regulating  the  manoeuvre 
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e.g.  by  accelerating  or  stopping  the  manoeuvre. 

To  prevent  the  system  from  moving  between  different  actions  very  often,  the  currently 
selected  action  is  given  high  priority  which  can  be  overrun  only  by  very  urgent 
situational  requirements. 

Motoric  Execution  of  an  Action 

The  chosen  action  sequence  can  either  initiate  an  operation  like  steering,  braking, 
stepping  on  the  gas  pedal  or  handle  blinker  and  flash  or  a  more  complex  motor  program 
according  to  the  theory  of  generalized  motor  programs  (Schmidt,  1976,  85),  Motor 
programs  like  lane  change  or  overtaking  are  actions  composed  of  a  well  defined  sequence 
of  discrete  operations.  Usually  these  motor  programs  have  been  internalized  by  the 
driver  through  long  time  training  and  are  executed  in  such  a  way  that  movements  are 
time  optimal.  Nevertheless  the  driver  controls  execution  and  regulates  if  the  result  of  the 
action  does  not  conform  to  his  expectations.  In  fig.  2  a  typical  diagram  of  the  lateral 
deviation  for  a  lane  change  with  corresponding  steering  wheel  angle  and  steering  wheel 
rate  of  a  well  trained  driver  is  shown  (Jiirgensohn,  1997), 

Motor  programs  and  discrete  actions  have  a  finite  duration  and  cannot  be  interrupted 
arbitrarily.  If  there  is  the  need  to  switch  to  an  action  that  is  more  urgent  than  the  current 
action,  though,  the  preferred  action  cannot  be  started  immediately,  but  only  after  a 
somewhat  delayed  interruption  of  the  actual  motor  program  due  to  neuromuscular  lag. 
While  in  general  actions  are  selected  by  explicit  considering  the  alternatives,  some 
actions  are  executed  skipping  the  decision  process,  for  example  emergency  braking  in 
panic  (reflex  action). 


Figure  2:  Schematic  motor  program  of  steering  wheel  angleX,  steering  wheel  rate 
VX  and  the  resulting  lateral  deviation  q  for  lane  change 

In  order  to  have  the  model  act  in  a  way  similar  to  human  drivers,  the  driver  model  is 
supplied  with  simplified  parametric  motor  schemes  which  have  been  observed  at  real 
drivers.  Parameters  are  current  speed,  acceleration  and  relative  distances.  Both  of  the  two 
action  sequences  available  to  the  driver  model  are  complex  motor  programs  composed  of 
other  motor  programs  and  several  single  vehicle  operations.  „Following“  includes 
„approaching“,  “braking“  and  „keeping  the  distance"  as  sub -procedures.  „Overtaking"  is 
composed  of  „change  to  left  lane",  „acceleration"  and  „change  to  right  lane". 
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Motivation 

As  already  mentioned,  the  psychological  state  of  the  driver  is  relevant  for  every  stage  of 
the  cognitive  process.  In  the  presented  driver  model  only  a  limited  number  of 
motivational  factors  are  accounted  for,  namely 

•  hurry 

•  competence 

•  daring 


While  plausible  assumptions  how  these  attributes  are  connected  with  human  behaviour 
are  quite  self-evident,  valid  quantitative  interdependences  will  be  almost  impossible  to 
formulate.  Here  a  fuzzy-like  description  using  linguistic  variables  could  be  considered, 
but  it  is  also  possible  to  presume  simple  functional  relationships.  (Domer  &  Hille,  1995) 
employ  very  elementary  functions  to  simulate  the  influence  of  emotion  and  motivation 
on  human  acting  in  „artificial  souls“  and  are  able  to  produce  reasonable  behaviour. 

For  example,  the  desired  speed  (the  overall  goal)  will  increase  with  the  individual’s 
hurry,  its  estimate  of  own  driving  skills,  the  degree  of  boldness  and  last  but  not  least  the 
type  of  vehicle.  It  is  further  assumed  that  estimation  of  safety  margins  and  even 
perception  of  e.g.  relative  speeds  are  dependent  on  motivational  factors.  Because  the 
evaluation  of  benefit  and  risk  is  also  subjective,  the  decision  process  is  thus  very 
sensitive  to  the  disposition  of  the  individual. 

Intersocial  Behaviour 

Intersocial  behaviour  can  show  itself  in  communication  activities  like  operating  blinker 
or  flash  or  driving  characteristics  like  jostling.  In  any  case  intersocial  behaviour  will  give 
drivers  hints  about  other  driver’s  intentions  and  a  means  to  communicate  own  plans. 
Usually  drivers  observe  other  traffic  participants  carefully  in  order  to  predict  their  future 
actions  and  be  able  to  react  in  an  appropriate  manner.  This  need  for  anticipation  requires 
that  the  driver  model  be  able  to  infer  other  drivers’  current  goals  and  intentions  from 
observed  actions,  in  other  words,  it  should  be  able  to  generate  an  internal  model  about 
the  mental  state  of  other  road  users.  As  a  typical  example  of  internal  models  the  frequent 
classification  according  to  distinctive  features  into  categories  like  „speedster“, 
„slowcoach“  or  „dare  devir‘  which  possess  typical  driving  patterns  can  be  mentioned. 

The  Simulation  Environment 

The  drivers  move  in  an  artificial  environment  consisting  of  the  road  and  fixed  or  moving 
objects  herein.  The  course  of  the  road  is  constructed  by  joining  discrete  street  segments 
together.  Objects  can  be  positioned  along  the  road  and  be  assigned  velocities  and 
trajectories  to  move  along.  The  whole  environment  can  be  configured  for  different 
driving  situations  by  defining  the  course  of  the  road  and  the  participating  objects  with 
their  characteristic  movements.  In  addition  it  is  possible  to  incorporate  a  human  driver 
into  the  situation.  The  simulation  environment  becomes  then  a  driving  simulator  and  can 
serve  to  validate  the  driver  model. 

Vehicles  are  manipulated  by  the  driver  model  by  operating  gas,  brake,  steering  wheel, 
blinker  and  flash,  whose  current  values  are  fed  into  a  single  track  model  of  the  vehicle  to 
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calculate  relevant  variables  of  state  like  position,  velocities  and  acceleration  (longitudinal 
and  transversal)  for  each  time  step. 


Figure  3  :  Schematic  representation  of  the  simulation  program 

Example  Driving  Situation 

The  driver  model  is  specified  for  the  investigation  of  the  situation  „overtaking  on  a  dual 
track  highway**,  see  figure  4,  At  least  4  vehicles  are  necessary  to  describe  this  situation, 
which  do  not  need  the  same  degree  of  model  accuracy,  though.  The  following 
assumptions  and  simplifications  are  made  for  the  described  driving  situation: 

•  the  leading  vehicle  (FI)  drives  at  fixed  speed  on  the  rightmost  lane, 

•  the  second  vehicle  follows  FI ,  but  starts  overtaking  with  a  given  probability, 

•  the  third  vehicle  (EGO)  contains  the  discussed  cognitive  driver  model.  EGO  decides 
according  to  the  perceived  traffic  situation,  its  motivation  and  its  estimation  of  the 
behaviour  of  the  other  drivers  between  two  cardinal  driving  programs,  following  or 
overtaking.  Both  driving  programs  consist  of  a  complex  set  of  action  sequences  (e.g. 
lane  change  or  keeping  a  constant  distance).  If  a  situation  rated  as  dangerous  emerges 
(e.g.  sudden  swerve  out  of  the  previous  vehicle  or  close  drive  up  of  the  car  behind)  the 
overtaking  manoeuvre  can  be  eventually  interrupted.  There  is  social  interaction  with 
vehicle  F4  that  approaches  quickly  from  behind  on  the  left  lane. 

•  the  vehicle  on  the  left  lane  (F4)  approaches  the  column  from  behind  at  high  speed. 
According  to  its  motivation  the  model  cooperates  with  EGO  or  not.  Decisions  are 
possible  only  with  respect  to  speed  choice  and  communication,  lane  change  has  not  to 
be  considered. 
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EGO  F2 


Figure  4  Investigated  traffic  situation 

The  described  driving  situation  allows  the  simulation  of  many  traffic  situations  due  to 
different  initial  conditions  and  driver  characteristics.  The  following  constellations  were 
considered  quite  illustrative  for  the  behaviour  of  the  driver  model: 

( 1 )  large  distance  between  F2  and  EGO,  EGO  not  motivated  to  drive  fast 

(2)  medium  distance  between  F2  and  EGO,  F2  is  driving  slowly,  both  EGO  and  F4 
are  motivated  to  drive  at  maximum  speed  in  sport  cars 

(3)  same  as  (2),  but  EGO  driving  a  slow  car 
Discussion  of  Results 

In  constellation  (1)  EGO  chooses  not  to  overtake,  but  first  accelerates  until  the  safety 
distance  is  almost  reached,  then  the  regulation  program  „follow  in  constant  distance“  is 
started  and  continued. 

Figure  5  shows  the  simulated  following  behaviour  plotted  in  the  Ax -Ax-  plane.  The 
parabolic  parts  of  the  curve  reflect  the  rule  that  whenever  the  model  chooses  to  approach 
or  to  brake,  it  does  so  with  constant  acceleration  or  deceleration  of  the  vehicle 
respectively.  The  irregular  distribution  of  turning  points  is  due  to  the  stochastically 
varying  perception  of  relative  distance  and  velocity.  Observed  car  following  behaviour 
results  in  curves  like  those  displayed  in  figure  6,  which  are  in  principle  similar  to  the 
calculated  ones. 
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Figure  5 :  Simulated  Following  Behaviour  in  the  Ax- Ax-  plane 


Figure  6 :  Observed  limit  cycles  of  car  following  in  the  Ax  -  Ax-  plane  (modified  after 
Hartwich,  1971) 

Constellation  (2)  is  more  interesting  with  respect  to  decision  making.  EGO  first 
approaches  and  then  decides  to  accelerate  with  maximum  gas  for  overtaking  in  order  to 
be  able  to  achieve  the  desired  speed  and  avoid  the  conflict  with  F4,  who  is  approaching 
quickly  and  operating  the  flash.  Due  to  the  stochastic  component  which  is  introduced 
through  perception  errors  further  development  of  the  situation  is  not  definitely 
determined.  If  the  estimation  of  relative  speed  and  distance  to  F4  was  on  the  safe  side, 
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overtaking  can  be  finished.  If  not  so,  or  if  F2  suddenly  swerves  out,  an  accident  can 
occur. 

The  third  case  demonstrates  the  anticipation  of  other  traffic  participants’  behaviour. 
Although  the  circumstances  are  equal  to  that  of  the  previous  example,  F4  brakes  much 
earlier  and  more  vigourously  supposing  that  EGO  shows  no  inclination  to  yield  and  that 
the  maximum  speed  of  EGO’S  vehicle  is  very  much  under  F4’s  speed. 

Conclusion 

A  cognitive  driver  model  that  accounts  for  motivation,  decision  making  and  goal  directed 
acting  as  well  as  human  perception  characteristics  and  motor  processes  with  emphasis  on 
decision  making  has  been  developed  in  order  to  simulate  critical  traific  situations.  The 
assertion  that  the  motivational  disposition  of  drivers  affects  traific  situations  to  a  great 
extent  could  be  confirmed.  First  results  show  that  the  chosen  modelling  method  using  a 
modified  production  system  is  suitable  for  the  purpose  to  simulate  individual  car  driving 
behaviour. 

For  an  assessment  of  novel  technologies  like  driver  assistance  systems,  however,  a  more 
realistic  modelling  of  perception  and  information  processing  will  be  necessary.  For  this 
objective  modelling  tools  like  ANNs  or  Fuzzy  could  be  employed. 
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Abstract 

Saturated  capacities  in  traffic  systems  evoke  increasing  interest  in  simulations  of  complex 
networks  serving  as  laboratory  environment  for  developing  management  strategies. 
Especially  for  urban  areas  questions  concerning  overall  traffic  control  have  to  be 
considered  with  regard  to  their  impacts  on  the  whole  network.  Modelling  traffic  flow 
dynamics  using  cellular  automata  allows  to  run  large  network  traffic  simulations  with 
only  comparative  low  computational  efforts.  We  present  a  traffic  simulation  tool  for 
urban  road  networks  which  is  based  on  the  Nagel-Schreckenberg  Model.  Arbitrary  kinds 
of  roads  and  crossings  are  modelled  as  combinations  of  only  a  few  basic  elements. 
Furthermore  parking  capacities  are  considered  as  well  as  circulations  of  public  transports. 
The  vehicles  are  driven  corresponding  to  route  plans  or  at  random  depending  on  the 
available  data.  The  application  of  this  network  simulation  covers  investigations  on  the 
field  of  traffic  planning  as  well  as  online  simulations  based  on  real-time  traffic  data  as 
basis  for  dynamic  traffic  management  systems. 

Introduction 

While  for  global  aimed  questions  in  the  field  of  traffic  management  like  dynamic  vehicle 
routing  it  is  self-evident  to  be  considered  on  the  whole  network,  even  local  aspects  like 
changes  in  priority  regulation  at  single  crossings  have  to  be  investigated  on  large  scales 
to  avoid  undesired  global  effects  due  to  mutual  correlations.  Testing  control  strategies  in 
realtiy  is  usually  infeasible  or  at  least  extremely  demanding  in  time  and  costs;  therefore 
the  application  of  simulation  tools  as  laboratory  environment  is  desired.  Designing  a 
simulation  tool  for  network  traffic  requires  to  find  an  overall  compromise:  To  enable 
collection  of  individual  vehicle  data  like  travel  times  or  number  of  stops  due  to  priority 
rules  a  microscopic  approach  is  appropriate.  In  addition  one  is  engaged  to  minimize 
running  times,  since  e.g.  for  checking  out  different  vehicle  routing  or  traffic  light  control 
strategies  complex  scenarios  have  to  be  simulated  again  and  again  such  that  large 
computation  times  are  very  cumbersome  and  inefficient.  Furthermore  for  online 
simulations  based  on  real-time  traffic  data  run  speeds  of  at  least  real-time  are  a  necessary 
requirement. 

We  present  a  microscopic  simulation  tool  for  vehicular  traffic  in  urban  road  networks  [1], 
which  was  developed  in  the  framework  of  the  Northrhine  Westfalia  Cooperative  FVU 
(Forschungsverbund  Verkehrssimulationen  und  Umweltwirkungen)  [2].  The  underlying 
traffic  flow  dynamics  is  based  on  the  cellular  automaton  introduced  by  Nagel  and 
Schreckenberg  [3].  This  approach  has  proved  as  very  efficient:  Meanwhile  it  is  possible 
to  simulate  even  the  whole  German  autobahn  network  in  real-time  [4,5]  and  within  the 
project  TRANSIMS  [6]  microscopic  traffic  simulations  are  carried  out  for  the  Dallas  / 
Fort  Worth  area  [7], 
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The  Cellular  Automaton  Model 

The  Nagel- Schreckenberg  model  [3]  was  originally  defined  on  a  single-lane  road.  The 
road  is  subdivided  into  cells,  which  can  be  either  empty  or  occupied  by  one  vehicle. 
Every  vehicle  has  a  non-negative  integer  velocity.  For  one  update  of  the  road  the 
following  four  steps  are  performed  simultanously  for  all  vehicles: 

1 .  Acceleration:  v  =  min(v  +  1 ,  Vmax ) 

2.  Avoiding  crashes:  v  =  min(v,  gap) 

3.  Randomization:  if  rand()  <  p  then  v  =  max(v  -1,0) 

4.  Update:  Each  vehicle  is  advanced  v  cells. 

Here  gap  denotes  the  number  of  empty  cells  in  front  of  a  vehicle  and  Vmax  the  maximum 
velocity.  The  randomization  (Step  3)  takes  into  account,  that  individual  driving 
behaviours  for  different  vehicles  result  in  non-deterministic  dynamics  of  vehicle  motions 
in  reality.  The  resolution  is  taken  to  be  7.5m  per  cell  representing  the  average  space  a 
vehicle  takes  in  a  jam,  but  can  be  suitably  adjusted  with  regard  to  the  considered  problem 
[8];  using  this  cell  width  each  time  step  in  the  simulation  corresponds  to  one  second  in 
reality.  For  modelling  traffic  on  multi -lane  roads  a  set  of  rules  for  lane  changes  is  added 
to  the  fundamental  rules  [9,10,11]. 

Detailed  investigations  showed  that  despite  its  simplicity  the  cellular  automaton  model  is 
capable  to  reproduce  macroscopic  traffic  flow  features  including  realistic  lane  changing 
behaviours  [9,12].  Also  a  continuous  extension  [13]  of  the  basic  discrete  model  and  a 
modified  set  of  rules  resulting  in  more  realistic  breaking  and  accelerating  behaviour  [14] 
was  developed  to  enable  modelling  of  additional  more  detailed  vehicle  features  (e.g. 
exhaust  emission). 

Network  Simulation  Tool 

The  road  network  is  described  as  composition  of  nodes  and  edges  representing  crossings 
and  roads.  To  avoid  misunderstandings:  Whenever  we  use  the  expression  "edge"  we  refer 
to  directed  edges  representing  one  direction  of  motion  on  a  road;  i.e.  one  road  usually 
consists  of  two  (oppositely  directed)  edges.  The  basic  idea  is  to  define  different  types  of 
edges.  This  approach  causes  arbitrary  complex  crossing  types  to  be  represented  as 
combinations  of  at  most  three  basic  edge  types:  On  single-lane  edges  simply  the  basic 
cellular  automaton  is  applied.  In  addition  there  can  be  connected  turn  pockets  for  each 
direction  at  the  end  of  the  edge.  Especially  at  the  end  of  multi-lane  roads  the  lane 
changing  behaviour  strongly  depends  on  the  desired  driving  direction.  For  this  purpose, 
multi-lane  edges  are  subdivided  into  different  regions.  Finally  also  highways  play  an 
important  role  for  a  realistic  description  of  urban  traffic.  Therefore  an  additional  element 
is  incorporated  to  simulate  highway  drive-ups  and  also  crossings  with  large  spatial 
extension.  Transfer  edges  are  one-  or  multi-lane  edges  which  merge  in  a  destination  edge. 
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Figure  1:  Representation  of  a  node  as  combination  of  different  edge  types. 

Fig.  1  shows  a  crossing  as  combination  of  different  edge  types.  The  numbers  at  the  end  of 
the  edges  represent  signal  groups  of  the  corresponding  traffic  lights  and  will  be  used 
below.  Optionally  the  first  cells  on  a  edge  can  be  defined  as  crossing  section  (shaded  in 
the  figure)  to  model  the  spatial  extension  roughly.  In  this  case  for  each  possible  direction 
it  is  predefined  if  vehicles  entering  the  edge  are  put  onto  the  first  cell  of  the  edge  or 
beyond  the  crossing  section.  This  also  allows  to  consider  cases  of  jammed  crossing 
regions  in  combination  with  priority  rules. 

One  striking  feature  of  urban  road  networks  is  the  high  density  of  intersections,  which 
causes  essential  correlations  among  traffic  streams  on  different  edges.  It  is  important  for 
the  simulation  to  have  reahstic  throughputs  at  the  crossings,  which  again  requires  to 
consider  complex  priority  rules  and  especially  traffic  lights  in  a  realistic  manner.  Each 
edge  in  the  network  has  a  driving-direction- dependent  leave  flag  at  its  end,  by  which 
vehicles  may  be  prevented  from  advancing  any  further.  Traffic  lights  are  reahzed  by 
switching  these  flags  corresponding  to  predefined  time  tables.  In  addition  a  hierarchy  can 
be  defined  to  consider  crossings  where  special  directions  are  controlled  by  more  than  one 
light  (e.g.  occasionally  active  green-light  arrows  for  right  turns).  To  facilitate  handling  of 
traffic  lights  the  simulation  tool  contains  an  editor  for  setting  green  and  red  phases  within 
a  switch  matrix. 

Priority  rules  are  realized  by  switching  leave  flags  depending  on  vacant  cells  on  other 
edges.  In  reality  turning  vehicles  are  often  hindered  in  driving  further  e.g.  by  pedestrians 
crossing  the  destination  road,  which  can  reduce  the  throughput  at  certain  crossings 
essentially.  This  effect  is  in  case  considered  by  blocking  turning  vehicles,  for  which 
driving  on  is  allowed  by  traffic  lights  and  priority  rules,  with  predefined  probabilities. 
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Furthermore  no-stop  and  stop  priority  rules  are  distinguished;  for  the  latter  it  is 
additionally  checked  if  vehicles  had  zero  velocity  standing  on  the  last  cell  of  the  edge  for 
at  least  one  time  step  before  they  are  allowed  to  drive  on. 

One  of  the  fundamental  questions  with  regard  to  network  traffic  simulations  concerns  the 
question  how  the  vehicles  are  guided  through  the  network.  In  a  randomly  driven 
simulation  at  each  crossing  destination  directions  for  the  vehicles  are  choosen  at  random 
corresponding  to  turn  counts.  Running  the  simulation  route  plan  driven  means  that  each 
vehicle  makes  its  turning  decision  corresponding  to  individual  route  plans,  which  are 
derived  from  origin-destination  information.  In  the  simulation  the  guidance  mode  is 
specified  for  each  vehicle  separately.  Combination  of  both  for  example  allows  to  check 
routing  strategies  on  random  background  traffic  representing  typical  traffic  states. 

The  vehicles  are  characterized  by  their  maximum  speed,  length  (number  of  occupied 
cells)  and  a  probability  for  carrying  out  risky  lane  changes  (i.e.  lane  changes  without 
considering  safety  distances).  In  the  actual  version  the  deceleration  probabilities  are  not 
individual,  but  attributed  to  the  edges  to  tune  throughputs  at  the  crossings.  There  are  two 
special  kinds  of  vehicles:  Vehicles  can  be  guided  periodically  along  predefined  routes 
following  time  tables  to  simulate  public  transports.  Additionaly  bus  respectively  tram 
stops  are  defined,  where  these  have  to  wait  for  a  certain  time  and  in  case  are  taken  out  of 
the  network  for  that  time.  The  second  type  covers  hindrances  like  accidents  or  road 
works.  These  are  represented  by  vehicles  of  special  length  and  zero  maximum  velocity. 

An  essential  characteristic  of  urban  traffic  is  the  fact  that  there  are  no  predefined  sources 
and  sinks;  rather  vehicles  are  allowed  to  enter  or  leave  the  network  at  nearly  every 
arbitrary  position.  For  that  reason  sources  and  sinks  can  be  linked  to  every  cell  in  the 
network.  At  sources  vehicles  are  characterized  by  type,  guidance  mode  (see  section 
Vehicle  Guidance )  and  -  if  necessary  --  additional  information  like  if  the  vehicle  follows 
a  dynamic  routing  system  or  not.  They  are  inserted  corresponding  to  source  rates  or  trip 
plans;  the  latter  means  that  at  the  beginning  a  list  of  vehicles  with  departure  times, 
vehicle  characteristics  and  route  plans  is  prepared.  In  addition  flow  check  points  as 
source-sink  combination  can  be  defined.  These  can  be  located  everywhere  in  the  network 
to  adjust  the  number  of  vehicles  to  counting  loop  data.  For  these  check  points  statistics 
covering  number  of  added  and  deleted  vehicles  are  collected  separately.  Route  plan 
driven  vehicles  are  taken  out  of  the  network  after  reaching  the  destination  node. 
Probabilities  for  leaving  the  network  at  the  next  node  are  attributed  to  every  edge  for 
randomly  driven  vehicles. 

Especially  for  urban  areas  the  consideration  of  parking  capacities  is  desired.  In  the 
simulation  these  are  considered  as  special  source- sink  pairs,  which  are  fiUed  up  with 
regard  to  predefined  capacities  and  from  which  —  as  far  as  they  are  not  empty  —  vehicles 
can  enter  the  network.  This  allows  to  develop  strategies  for  parking  control  systems.  It  is 
also  possible  to  define  parking-lanes  on  the  roads,  which  are  used  as  usual  lanes  and  in 
addition  vehicles  can  be  put  on  them  marked  as  parking.  This  feature  allows  to  consider 
capacity  reduction  of  roads  due  to  parking  vehicles. 

A  necessary  requirement  for  traffic  management  systems  is  information  about  the  traffic 
state.  For  this  purpose,  three  instruments  are  incorporated  in  the  simulation:  At  measure 
points  local  data  like  number  and  average  speed  of  vehicles  are  collected  separately  for 
different  vehicle  types  and  averaged  over  predefined  time  intervals.  Edges  can  also  be 
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grouped  together  representing  measure  regions  ,  by  which  global  measurings  are  carried 
out.  The  individual  travel  time  is  measured  for  every  vehicle  in  the  network.  For  more 
detailed  information  it  is  possible  to  guide  vehicle  probes  through  the  network.  This 
allows  to  collect  additional  point  to  point  data  for  special  routes  (e.g,  data  with  regard  to 
the  driving  comfort  like  number  and  duration  of  stops  due  to  priority  rules  and  traffic 
lights).  Furthermore  overall  network  data  like  number  of  vehicles,  average  speed  and 
edge  usage  are  stored. 

The  overall  simulation  tool  consists  of  two  principal  processes.  The  master  controller 
maintains  the  overall  coordination:  it  checks  static  and  dynamic  network  data  read  from 
the  database  for  consistence  and  initializes  the  scenarios.  During  the  simulation  it 
receives  and  updates  time  dependent  data  like  turn  counts,  handles  the  simulation  output 
including  in  case  updates  of  the  graphics  and  if  necessary  provides  dynamic  data  to  the 
routing  module,  which  updates  route  plans  for  the  vehicle  guidance  system.  The  actual 
network  dynamics  is  carried  out  by  the  micro  simulation  process;  i.e.  vehicle  motions, 
source  and  traffic  light  updates  and  data  collection  for  statistics.The  advantage  of  this 
subdivision  is  twofold:  On  the  one  hand  it  speeds  up  the  simulation,  since  in  addition  the 
micro  simulation  can  be  parallelized  independently.  Furthermore  in  practice  it  facilitates 
overall  handling,  because  it  is  possible  to  rearrange  e.g,  graphical  output  or  data  formats 
without  caring  about  the  complex  structure  of  the  micro  simulation. 

Simulation  of  the  Duisburg  road  network 

Currently  traffic  simulations  for  the  inner  city  of  Duisburg  are  developed  applying  the 
simulation  tool.  Here  we  focus  on  presenting  the  present  traffic  state  using  online  traffic 
data  stemming  from  induction  counting  loops  provided  by  the  traffic  control  center  of  the 
municipality  and  updated  every  minute. 
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Figure  2:  The  simulated  road  network  of  Duisburg. 


Fig,2  shows  the  considered  network  covering  107  nodes,  280  edges;  the  total  lane  length 
amounts  to  about  165  kilometers  (22059  cells).  Inclusive  online  source  and  turn  count 
updates,  priority  regulation  and  data  collection  for  statistics  it  is  possible  to  simulate  in  20 
minutes  on  a  PC  (Pentium  P133)  a  whole  day  of  typical  traffic. 


Time 


Figure  3:  Number  and  average  speed  of  vehicles  in  the  Duisburg  network  during  an 

online  simulation. 


Fig.3  shows  the  number  of  vehicles  and  the  average  speed  during  an  online  simulation 
for  the  time  interval  from  12.23am  on  the  1st  until  12.22am  on  the  2nd  of  April  in  1997. 
The  extended  minimum  in  the  night  and  the  two  maxima  at  about  Sam  and  5pm  are  well 
dicemible.  Due  to  missing  origin-destination  information  with  appropriate  time  resolution 
the  simulation  is  driven  randomly,  while  turn  counts  are  derived  from  real-time  traffic 
data.  The  present  traffic  state  for  the  city  of  Duisburg  is  presented  in  the  Internet  [15]. 

Conclusions 

We  presented  a  simulation  tool  for  urban  traffic.  The  microscopic  dynamics  based  on  the 
Nagel-Schreckenberg  cellular  automaton  allows  to  simulate  large  networks  in  multiple 
real-time.  Within  the  network  model  complex  crossings  inclusive  realistic  traffic  lights 
and  priority  rules  are  considered  as  well  as  parking  capacities  and  circulations  of  public 
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transports.  In  combination  with  real-time  traffic  counts  this  tool  serves  as  useful 
laboratory  environment  for  designing  and  checking  dynamic  traffic  management  systems. 
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Intelligent  Speed  Adaptor  (ISA) 
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This  manuscript  concerns  a  study  on  the  effects  of  a  prototype  intelligent  speed  adaptor 
(ISA)  in  actual  traffic.  Twenty-four  subjects  were  included  in  a  test  of  effects  of  feedback 
on  speed  behaviour,  mental  workload  and  acceptance.  Subjects  drove  an  instrumented 
vehicle  in  normal  traffic  on  various  types  of  roads  with  different  speed  restrictions  and  in 
a  driving  simulator,  interacting  with  other  traffic  as  well  as  under  different-  speed 
restrictions.  Subjects  completed  both  test  parts  twice,  half  of  the  subjects  receiving 
feedback  in  the  second  trial  (experimental  group),  half  of  the  subjects  not  (control 
group).  The  groups  differed  in  several  ways,  the  most  important  being  adaptation  of  their 
behaviour  under  feedback.  Subjects  in  the  experimental  group  behaved  more  according 
to  traffic  rules,  in  particular  speed  limits,  than  subjects  in  the  control  group.  No 
differences  in  workload  were  found.  Several  types  of  feedback  were  tested  to  acceptance 
and  rated  differentially. 

Introduction 

The  Dutch  Ministery  of  Transport  and  PubUc  Works  (‘AVV’,  Transport  Research 
Centre)  has  initiated  a  project  (‘Automatisering  Rijtaak’,  automatizing  the  driving  task) 
whose  objective  is  to  delineate  standardized  test  methodologies  for  assessing  safety 
effects  of  advanced  transport  telematics  (ATT).  This  report  presents  a  study  in  which  the 
general  methodology,  as  proposed  by  Verwey,  Brookhuis  and  Janssen  (1996),  is  followed 
to  investigate  the  effects  of  feedback  on  speed  behaviour,  mental  workload  and 
acceptance  of  telematics  applications  developed  to  regulate  speed. 

As  put  forward  by  Kuiken  and  Heijer  (1995)  in  their  theoretical  considerations  regarding 
driver  support  systems  and  traffic  safety  in  general,  this  type  of  systems  aims  to  facilitate 
driving  performance.  One  of  the  functionalities  of  such  a  system  may  be  to  facilitate  the 
task  pc  rformance  of  drivers  by  providing  real-time  advice,  instruction  and  warnings.  The 
latter  type  of  systems  are  usually  also  described  by  the  term  ‘co-driver  systems’.  Co¬ 
driver  systems  may  operate  in  advisory,  semi-automatic  or  automatic  mode  (e.g. 
Rosengren,  1995),  which  concern,  for  instance,  speed  regulation  systems.  Speed 
regulation  as  driver  support  system  per  se  is  different  from  speed  regulation  in  (fully) 
automatic  form,  such  as  in  autonomous  intelligent  cruise  control  (AICC)  which  is 
especially  developed  for  and  useful  on  motorways.  In  advisory  mode  a  speed  regulation 
system  is  suitable  for  any  environment  and  situation. 

According  to  Kuiken  and  Heijer  (1995)  the  underlying  assumption  behind  presenting 
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drivers  with  specific  information  about  the  driving  environment  or  supporting  them  in 
performing  certain  aspects  of  their  tasks,  is  that  their  behaviour  will  change  for  the 
better.  For  example,  if  you  warn  a  driver  that  his/her  speed  is  too  high,  s/he  will  decrease 
the  speed  (see  De  Waard  et  al.,  1994,  De  Waard  and  Brookhuis,  1995a),  Thus  far,  little  is 
known  about  the  effects  of  driver  information  and  support  systems  on  traffic  safety. 
Kuiken  and  Heijer  (1995)  recommended  to  develop,  at  the  first  stage,  guidelines  that  will 
allow  a  systematical  assessment  of  current  and  future  in-vehicle  support  systems.  In 
assessment  procedures  developed  so  far  (Verwey,  Brookhuis  and  Janssen,  1996)  the 
human  driver  is  considered  in  relation  to  the  traffic  environment,  in  terms  of  their 
capacities  and  limitations,  terms  of  the  tasks  that  need  to  be  performed,  and  in  terms  of 
the  errors  (i.e.  traffic  rule  violations)  they  might  make. 

Violation  of  traffic  rules  is  obviously  unwanted  because  of  its  relationship  to  accident 
causation.  Well-known  deviations  from  the  norm,  related  to  accident  causation  are  close 
following,  driving  while  intoxicated,  speeding,  running  red  traffic  lights  and  neglecting 
stop  signs  (cf.  Smiley  and  Brookhuis,  1987,  Rothengatter,  1991).  Speeding  has  a 
relatively  well-known  relationship  to  accident  involvement.  Salusjarvi  (1981)  reported  an 
almost  linear  relationship  between  change  in  number  of  accidents  and  change  of  mean 
speed  in  Finland.  Joksch  (1993)  found  that  the  probability  that  a  driver  is  killed  increases 
with  increased  speed  fits  regression  models  with  exponents  as  high  as  four,  meaning  that 
each  km/h  faster  exponentially  increases  fatality  risk.  An  average  reduction  of  as  little  as 
2  km/h  to  5  km/h  could  lead  to  a  reduction  of  10%  up  to  30%  in  injury  accidents  (for  a 
more  elaborate  overview,  see  Rothengatter,  1993).  However,  these  effects  seem  less 
evident  on  motorways  where  the  maximum  speed  (120  km/h  in  the  Netherlands)  is 
allowed  than  on,  for  instance,  rural  roads  where  a  speed  limit  of  80  km/h  is  imposed 
(Wegman  et  al.,  1991).  One  approach  to  influencing  unwanted  traffic  rule  violations,  in 
particular  speeding,  is  embedded  in  a  project  in  the  European  DRIVE  (Dedicated  Road 
Infrastructure  for  Vehicle  safety  in  Europe)  initiative,  the  DETER  (Detection, 
Enforcement  &  Tutoring  for  Error  Reduction)  project.  DETER  is  concerned  with  the 
development  and  testing  of  prototypes  of  on-site,  in-vehicle  and  integrated  monitoring, 
tutoring  and  enforcement  systems,  among  which  Intelligent  Speed  Adaptation  (ISA,  see 
also  De  Waard  and  Brookhuis,  1995b). 

Intelligent  Speed  Adaptation  (ISA) 

Through  recent  legislation  in  the  Netherlands,  the  maximum  driving  speed  is  restricted 
by  a  speed  limiter  in  the  heavier  types  of  lorries  and  coaches.  As  a  consequence  of  this 
legislation  the  number  of  heavy  vehicles  in  which  the  maximum  driving  speed  is 
restricted  will  only  increase  in  the  coming  years  (Alink,  1992).  The  effect  of  these 
devices  on  fuel  consumption,  noise,  air  pollution,  wearing  of  the  tires  and  traffic  safety  is 
expected  to  be  mainly  positive  (e.g.,  Almqvist  et  al.,  1991,  Van  der  Mede,  1992).  The 
obvious  restriction  of  the  speed  limiter  as  mandatory  now,  is  that  it  only  prevents  driving 
above  ^  maximum  allowed  driving  speed  of  heavy  goods  haulage  vehicles,  and  is 
independent  of  local  limit  in  a  specific  road  environment.  An  intelligent  speed  adapter 
(ISA)  takes  into  account  local  restrictions,  and  adjusts  the  maximum  driving  speed  to  the 
posted  maximum  speed.  When  it  comes  to  restriction  of  driving  speed  of  private 
vehicles,  the  use  of  intelligent  speed  limiters  is  to  be  preferred  due  to  further 
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differentiation  of  speed  limits  for  private  cars  compared  to  heavy  goods  vehicles.  A  non- 
intelligent  speed  limiter  is  set  at  the  maximum  allowed  driving  speed  for  motorways  (120 
km/h),  while  the  majority  of  a  speed  limiting  system’s  safety  benefits  can  be  attained  on 
‘A’ -class  roads  (limit  80  km/h)  and  in  built-up  areas  (50  km/h). 

Intelligent  speed  limiters  require  vehicle-infrastmcture  communication.  For  exchange  of 
information  about  the  road  class  or  local  restrictions,  one  option  is  to  equip  traffic  signs 
with  transmitters,  beacons  or  tags.  At  the  moment  a. car  passes  such  a  sign,  the  new  speed 
limit  is  to  be  conveyed  to  the  vehicle  in  one  way  or  the  other.  Examples  of  experimental 
car-infrastructure  communication  can  be  found  in  Sweden  (Nilsson  and  Berlin,  1992, 
Palmquist,  1993,  Persson  et  al.,  1993)  and  the  Netherlands  (De  Waard  and  Brookhuis, 
1995a,b). 

In  general,  a  standard  speed  limiter  is  an  intrusive  system  that  restricts  speed  control,  i.e, 
the  device  sets  the  maximum  possible  driving  speed.  An  intelligent  speed  limiter  is  able 
to  set  this  maximum  speed  in  accordance  with  local  posted  legal  limits.  A  less  intrusive 
device  is  a  system  that  provides  the  driver  with  feedback  about  local  limits,  for  instance, 
on  the  accelerator.  An  active  gas  pedal  increases  the  counterforce  if  the  driver  is  driving 
too  fast  (Godthelp  and  Schumann,  1991).  In  principle,  such  a  speed  limiter  leaves  the 
driver  in  control,  while  the  feedback  provided  in  case  of  a  speed  violation  is  highly 
compelling.  Moreover,  the  feedback  is  provided  in  the  tactile  modality,  i.e.  the  same 
modality  through  which  action  has  to  be  undertaken  to  observe  the  rules  again.  Feedback 
could  also  be  presented  in  the  visual  modality,  e.g.  a  warning  light  or  message  in  the 
dashboard,  or  auditory,  an  acoustic  signal  or  vocal  message.  On  the  one  hand  this  type  of 
feedback  seems  less  intrusive  than  the  feedback  an  active  gas  pedal  provides  because 
these  warnings  can  easily  be  ignored.  On  the  other  hand,  it  might  be  that  the  social  effect 
of  being  warned  in  the  presence  of  other  passengers  is  a  more  severe  chastisement  and 
therefore  less  preferred.  Anyway,  acceptance  of  the  feedback  type  systems  can  be 
expected  to  be  higher  than  of  a  strict,  standard  speed  limiter,  because  behaviour  is  less 
restrained.  Results  from  a  questionnaire  survey  demonstrate  that  a  slight  majority  of 
people  consider  an  indicator  of  speed-limit  violations  useful,  while  only  35%  of  the 
respondents  were  of  that  opinion  with  respect  to  a  speed  limiter  (Hagen  and  Fokkema, 
1990).  Another  advantage  of  the  feedback  systems  opposed  to  the  speed  restricters,  is 
that  speed  violations  can  sometimes  be  advantageous  for  traffic  safety,  e.g,,  there  are 
instances  in  which  a  (perhaps  misjudged)  critical  overtaking  manoeuvre  is  faster  and 
safer  performed  if  the  limit  is  exceeded. 

Development  of  a  prototype  intelligent  speed  adaptor 

An  effort  is  now  undertaken  in  the  Netherlands  to  develop  a  prototype  intelligent  speed 
adaptor  that  leaves  the  driver  in  control.  For  a  start,  this  resulted  in  the  development  of  a 
continuous  feedback  display  in  close  proximity  of  the  speedometer  indicating  the  current 
speed  limit,  quite  similar  to  the  CAROSI  system  (Nilsson  and  Berlin,  1992).  Central  part 
of  the  CAROSI  (CAr  ROadside  Signalling)  system  is  the  instrument  panel,  which 
includes  not  only  standard  displays  such  as  the  speedometer,  but  also  contains  sections 
on  which  roadside  information  is  displayed.  Amongst  these  is  the  posted  speed  limit, 
which  is  displayed  below  the  speedometer.  Major  advantage  of  giving  feedback  by 
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displaying  the  speed  limit  inside  the  car  is  that  this  information  remains  continuously 
visible  instead  of  only  being  visible  at  the  moment  a  sign  is  passed.  This  might  reduce 
speeding  because  of  general  unawareness  of  the  limit,  which  is  not  unconunon  in  the 
Netherlands  (e.g.,  Steyvers  et  al.,  1992,  De  Waard  et  al.,  1995). 

A  special  version  of  the  latter  type  of  feedback  display  is  developed  for  implementation 
in  an  experimental  test- vehicle.  Whenever  the  speed  ,  limit  is  exceeded  the  colour  in 
which  the  speed  limit  is  displayed  changes  from  green  (‘normal/neutral’)  to  amber,  or 
yellow,  (‘warning*).  In  case  the  speed  limit  is  exceeded  by  10%  the  colour  changes  from 
amber  to  red  (‘violation^),  and  then  an  additional,  auditory  warning  message  is  issued 
(see  also  De  Waard  et  al.,  1994,  De  Waard  and  Brookhuis,  1995ab).  The  systems  are 
integrated  in  the  existing  DETER  system  (see  De  Waard  and  Brookhuis,  1995a),  which 
is  developed  as  an  open  system  to  integrate  driver  monitoring  and  feedback 
(sub) systems.  In  the  present  experiment  this  set-up  is  tested,  letting  subjects  drive  the 
test- vehicle  with  and  without  the  feedback  systems. 

Additionally,  an  active  gas  pedal  is  tested  as  a  medium  for  haptic  feedback  in  case  of 
speed  limit  violations,  exceeding  by  10%,  in  a  driving  simulator  with  the  same  subjects, 
in  a  cross-over  design.  All  modes  of  feedback  are  studied  to  effects  on  behaviour,  mental 
workload  and  acceptance. 

Method 

Subjects 

Twenty-four  subjects  were  selected  from  the  TRC  subjects’  pool.  They  were  paid  for  their 
participation  in  the  test  on  effects  of  feedback  concerning  speed  restrictions  and 
violations  in  the  institute’s  instrumented  test  vehicle  and  driving  simulator. 

Experimental  design  and  procedure 

Upon  arrival  at  the  institute,  subjects  were  first  informed  about  the  purpose  of  the  study 
in  general  terms,  and  more  specifically  what  was  expected  from  them.  Heart  rate 
electrodes  were  fixed  at  appropriate  places  on  the  chest.  Then  a  general  questionnaire 
concerning  personal  data,  such  as  driving  experience  and  their  ideas  about  speed 
restriction  systems,  was  completed  after  which  half  of  the  subjects  drove  the 
instrumented  test  vehicle  over  a  fixed  route  and  then  performed  the  simulator  test,  half  of 
the  subjects  vice  versa.  Each  of  the  test-rides  consisted  of  two  parts,  first  the  baseline 
measurement,  then  after  a  short  break,  either  the  test  ride  with  feedback  or  the  control 
ride.  Half  of  the  subjects  received  feedback,  half  were  in  the  control  condition.  The 
subjects  were  instructed  to  drive  as  they  would  normally  do,  in  their  own  cars,  and  were 
told  in  the  second  ride  that  behaviour  with  respect  to  law  compliance  was  monitored  and 
feedback  was  possibly  given. 

The  test  rides  in  the  instrumented  test  vehicle  were  in  normal  traffic,  under  various 
conditions.  Subjects  were  guided  by  sampled  vocal  route  guidance  messages  that  were 
triggered  by  the  investigator  for  reasons  of  proper  timing.  They  were  led  over  a  varied 
route  that  included  sections  of  motorways,  A-roads  and  built-up  areas,  with  speed 
restrictions  of  50,  70,  80,  100  and  120  km/h.  The  test  ride  in  the  simulator  was  a  copy  of 
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the  situation  used  for  the  DETER  experiment  (De  Waard,  1995a),  other  traffic  being 
present,  speed  restrictions  being  50,  80,  100  and  120  km/h. 

After  each  of  the  (four)  test  rides,  subjects  were  requested  to  complete  questionnaires 
concerning  perceived  workload  and  subjective  driving  quality.  At  the  end  of  the  whole 
test,  subjects  completed  a  general  questionnaire  again,  asking  for  their  ideas  with  respect 
to  speed  resticting  systems  again. 

Test  vehicle 

For  the  test  rides  on  the  road,  the  instrumented  test  vehicle  (a  Renault  19  RT)  was  used 
(for  a  description  of  the  instrumentation,  see  Kok  and  Brookhuis,  1995),  Data  regarding 
speed  and  steering  wheel  movements  were  sampled  on-line  at  10  Hz  by  an  industrial  PC 
with  a  Pentium™  processor  and  were  stored  on  harddisk.  Information  regarding  speed 
Limits  and  stop  signs  were  conveyed  to  the  car  by  means  of  a  microwave  system  adapted, 
not  especially  developed,  for  this  specific  purpose,  by  an  electronics  manufacturer.  An 
antenna  (MIDS  SAA-4S)  /  reader  (MIDS  VI)  combination  was  instaUed  on  top 
respectively  inside  the  vehicle,  while  traffic  signs  were  equipped,  at  the  back  side,  with 
tags  (labels,  MIDS  ML20).  This  microwave  system  operates  at  a  frequency  of  2.4-2.5 
GHz.  Positioning  of  the  tags  at  the  back  side  of  traffic  signs  ensured  that  information 
regarding  local  speed  limits  did  not  enter  the  vehicle  too  early,  i.e.  not  before  a  speed 
limit  zone  actually  was  entered.  In  this  way  it  was  avoided  that  drivers  were  ‘accused’  of 
speeding  by  the  system  when  the  new  restrictions  had  not  taken  effect  yet  in  geographical 
sense.  Feedback  about  local  speed  limits  was  provided  only  to  subjects  in  the 
experimental  condition  during  feedback  trials  (i.e.,  the  second  series  of  trials).  On  the 
road  the  display  indicating  the  speed  limit  in  green  (when  keeping  to  the  limit),  amber 
(when  exceeding  the  limit  but  less  than  10%)  or  red  (when  exceeding  the  limit  by  more 
than  10%).  The  vocal  message  ‘You  are  driving  too  fast,  the  local  limit  is  was  given 
at  the  instance  the  display’s  colour  changed  from  amber  to  red.  Headway  feedback  was 
not  given  in  this  experiment. 

Simulator 

Subjects  performed  two  rides  in  the  institute’s  driving  simulator  (Van  Wolffelaar  and 
Van  Winsum,  1992,  Van  Winsum  and  Van  Wolffelaar,  1993).  Subjects  drove  in  the 
same  condition  as  in  the  on-the-road  test,  i.e.  subjects  in  the  control  condition  did  not 
receive  feedback  regarding  detected  violations.  Subjects  in  the  experimental  condition 
were  provided  with  haptic  feedback  about  the  (posted)  speed  limit  when  exceeding  the 
limit,  but  this  feedback  was  given  in  the  second  trial  only.  Haptic  feedback  consisted  of  a 
continuous  counterforce  on  the  gas  pedal.  The  counterforce  was  dependant  upon  the 
extent  to  which  the  limit  was  exceeded,  and  was  equal  to:  4  x  (speed  -  speed  limit  (both 
in  m/s))  Newton.  If  the  speed  limit  was  exceeded  by  more  than  10%  the  same  vocal 
message  was  given  as  in  the  on-the-road  trials.  The  same  testbed  as  described  in  De 
Waard  et  al.  (1994)  was  used  with  one  exception:  in  the  previous  simulator  experiment 
the  last  3  minutes  of  driving  of  each  trial  consisted  of  tedious  dual-carriageway  driving. 

Subjective  measures 

During  the  whole  test,  performance  parameters  and  subjects'  heart  rate  were  registered  as 
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measures  of  workload,  whereas  after  each  ride  the  self-report  scale  RSME  (Rating  Scale 
Mental  Effort,  see  Zijlstra  and  Meijman,  1989)  was  completed.  In  previous  tests,  both 
performed  in  the  simulator  (De  Waard  et  al.,  1994)  and  on-the-road  (De  Waard  and 
Brookhuis,  1995a,b),  results  showed  that  the  tutoring  messages  increased  driver  mental 
load  in  a  moderate  way.  Both  reduced  heart  rate  variance  and  increased  self-report  scores 
indicated  increased  effort. 

A  subjective  driving  quality  score  (driving  quality  scale,  see  Brookhuis  et  al,  1985)  was 
also  derived  to  check  on  subject’s  personal  impression  about  driving  circumstances. 

Before  entering  the  experiment  and  after  the  whole  session,  a  score  on  nine  items 
regarding  acceptance  of  tutoring  and  enforcement  systems  (Van  der  Laan  et  al.,  1997) 
was  derived  from  the  subjects.  After  the  test  rides  more  detailed  questions  about 
acceptance  of  the  different  feedback  systems  were  posed. 

Results 

A  total  of  24  subjects  participated  in  the  test  rides.  Eleven  subjects  were  female,  average 
age  34.6  {sd  8.7),  average  mileage  10.000  km/year  {sd  9.000),  thirteen  were  male, 
average  age  34.7  {sd  10.7),  mileage  21.000  km/year  {sd  19.000).  On  average,  both  males 
and  females  had  held  a  drivers’  licence  for  15  years.  Five  subjects  did  not  (fully) 
complete  the  test  due  to  simulator  sickness,  but  were  as  much  as  possible  included  in  the 
analyses. 

Although  the  number  of  detected  violations  during  the  second  (feedback)  series  of  trials 
is  lower  in  the  experimental  group,  this  effect  does  not  attain  statistical  significance 
(Group  X  Trial  interaction:  F(l,18)  =  1.33,  NS).  The  difference  in  the  number  of  speed 
violations  between  the  two  test  facilities  is  significant  (F(l,18)  =  25.4, p<  0.001),  drivers 
more  frequently  violated  the  speed  limit  on-the-road.  The  extent  to  which  the  speed  limit 
was  exceeded  was  higher  in  the  simulator  (F(l,18)  =  24.3,  p<  0.001).  During  the 
feedback-trial  the  extent  to  which  the  limit  was  exceeded  was  on  average  lower  (with  the 
exception  of  the  second  on-the-road  trail  for  the  control  group),  but  this  effect  is  not 
significant  (Group  x  Trial  interaction  F(l,18)  <  1,  NS,  Group  x  Trial  x  Testfacihty 
interaction,  F(l,18)  =  2.7,  NS), 

During  the  test  rides  it  was  noted  that  some  of  the  subjects  in  the  experimental  group 
obviously  ‘played’  with  the  feedback  display  to  test  its  behaviour  or  use  it  to  stay  in  the 
marginal  area  (amber).  It  was  therefore  decided  to  determine  a  new  parameter,  the 
propordon  of  time  violating  the  limit.  This  parameter  does  not  discretely  sum  up  the 
number  of  times  the  limit  is  exceeded,  but  reflects  the  time  the  driver  is  not  complying. 
Two  parameters  were  determined,  the  proportion  of  time  driving  above  the  limit,  i.e.  the 
time  the  display  was  or  would  have  been  amber  or  red,  and  the  proportion  time  driving 
above  the  limit  -i-  10%,  i.e.  the  time  the  display  was  or  would  have  been  red  and  an 
auditory  message  was  or  would  have  been  issued.  The  "would  have  been'-condition  is  for 
the  control  group,  the  experimental  group  actually  received  the  described  feedback.  From 
figure  1  it  is  clear  that  as  much  as  20  to  25%  drivers  are  speeding  in  the  strict  juridical 
sense.  Between  5  and  10%  of  the  time  they  are  driving  faster  than  the  speed  limit  plus  a 
10%  margin.  The  effect  of  the  feedback  system  is  only  significant  for  the  latter  parameter 
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(Group  X  trial  interaction:  F(l,22)  =  1.50,  NS  for  amber/red  display,  F(l,22)  =  9.39, 
/7<0.01  for  red-display  feedback). 


Percentage  time  driving  above  limit 


1  2T  1  2T 

AMBER  or  RED  (>  LIMIT)  RED  (>  LIMIT  4-  10%) 


Figure  1  Time  driving  above  the  limit  and  above  the  limit  plus  a  10%  margin  per  group 
and  Trial  (on~the-road  data  only).  1  =  Trial  1,  2T  =  Trial  2  (T=  Tutoring  feedback, 
provided  to  subjects  in  the  experimental  group  only). 

Workload 

Task  performance  was  measured  by  the  vehicle  parameters  SD  of  the  steering  wheel 
movements  (on  straight  sections  in  the  road),  average  lateral  position  on  the  road  and  SD 
of  the  lateral  position  on  the  road.  During  the  second  simulator  trial  swerving  as 
measured  by  the  SD  of  the  lateral  position  increased  in  both  groups  (F(l,18)  =  7.59, 
p<0.05).  No  significant  changes  in  task  performance  parameters  as  a  result  of  the 
feedback  system  were  found. 

Self-report  ratings  on  the  Rating  Scale  Mental  Effort  (Zijlstra  and  Meijman,  1989)  show 
a  clear  effect  of  test  environment  (F(l,17)  =  26.6,  p<  0.001).  Previous  comparison  of 
trials  between  on-the-road  driving  and  simulator  tests  had  also  shown  increased  mental 
effort  for  task  performance  in  the  simulator  (De  Waard  and  Brookhuis,  1995a).  There  is  a 
tendency  to  evaluate  the  amount  of  invested  effort  in  the  second  trial  as  lower  (F(l,17)  = 
4.04,  /?<0.10).  The  interaction  Group  x  Trial  approaches  significance  for  the  car  data 
only  (F(l,22)=  3.98,p<0.06). 
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One  physiological  measure  was  registerd,  the  subjects’  heart  rate,  or  ECG.  Data  were  off¬ 
line  checked  for  artifacts.  Different  problems  varying  from  simulator  sickness  to  cable 
problems  led  to  incomplete  data  sets,  four  subjects  had  to  be  deleted  from  the  car  data 
set,  eight  subjects  from  the  simulator  set,  ECG  analysis  resulted  in  parameters:  IBI  (inter¬ 
beat-interval)  and  variation  coefficient  (normalized  heart  rate  variability  in  the  time 
domain).  Both  parameters  are  sensitive  to  general  arousal  and  effort.  There  were  no 
effects  of  feedback  on  any  of  the  heart  rate  parameters.  Only  self-reported  mental  effort 
during  the  trials  performed  on-the-road  are  indicative  for  a  slight  increase  in  mental 
workload.  Physiology  and  task  performance  did  not  reflect  any  changes  as  a  result  of 
feedback. 

Control  variables 

Control  variables  for  driving  performance  are  self-report  ratings  about  driving  quality 
and  average  and  sd  of  driving  speed.  After  each  trial  subjects  were  asked  to  compare 
their  performance  to  normal  driving  on  the  driving  quality  scale  (Brookhuis  et  al.,  1985). 
The  scale  has  a  range  from  -100  (T  have  driven  exceptionally  poorly’)  to  -i-lOO  (T  have 
driven  exceptionally  well’),  while  a  rating  of  ‘0’  stands  for  T  have  driven  normally’. 
Analysis  shows  an  interaction  between  trial  and  testfacility  (F(l,17)  =  9.1,  p  <  0.01). 
Rated  driving  quality  decreases  slightly  in  the  second  on-the-road  trial,  while  it  improves 
during  the  second  simulator  trial.  Probably  due  to  relatively  low  familiarity  driving 
quality  in  the  simulator  is  rated  lower  than  in  a  real  car  (F(l,17)  =  17,8, p  <  0.001). 

Speed  was  determined  on  the  motorway  (speed  limit  120  km/h),  on  a  rural  50  km/h 
speed-limit  road  (both  on-the-road)  and  in  the  simulator  on  a  dual  carriageway  (speed 
limit  100  km/h).  The  difference  between  trial  1  and  2  is  significant  for  both  average 
speed  (F(l,17)  =  6.0,  p  <  0,05)  and  variability  in  driving  speed  (F(l,17)  =  10.4,  p  < 
0.01). 

The  control  variables  show  that  rated  task  performance  as  well  as  average  driving  speed 
on  high  speed  roads  increase  with  time-on- task  (i.e.,  trials).  Driving  speed  and  variability 
(sd)  in  driving  speed  decrease  mostly  in  the  second  trial,  in  particular  in  the  experimental 
group. 

Ratings  on  the  acceptance  items  (Van  der  Laan  et  al,  1997)  did  not  differ  significantly 
between  the  control  and  experimental  group  (F(l,22)  <  1,  NS),  nor  did  they  differ 
between  before  and  after-test  measurements  (F(l,22)  <  1,  NS).  Usefulness  of  a  speed 
compliance  feedback  system  is  rated  negatively,  -1.04,  on  a  scale  from  -2  to  4-2.  Such  a 
system  is  rated  slightly  positive  in  terms  of  being  satisfying  (-1-0, 1 6)  on  a  scale  with  the 
same  range. 

After  the  test  rides,  subjects  from  the  experimental  group  were  asked  to  rate  the  effect  the 
feedback  system  had  had  on  their  speed  choice.  Evaluation  in  terms  of  good/bad  and 
pleasant/not  pleasant  are  given  in  table  1.  All  drivers  had  been  asked  whether  they  would 
accept  the  different  types  of  systems  The  display-only  turned  out  to  be  the  best  accepted 
system,  followed  by  haptic  pedal  feedback  (experimental  group)  and  auditory  feedback 
(control  group). 
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Table  1.  Evaluation  of  the  different  types  of  feedback  systems  (N^ll). 


Feedback 


‘Good’  ‘Not  Good’  ‘Pleasant’  ‘Not  pleasant’ 


Auditory  (car)  73% 

Visual  (Display)  (car)  91% 

Auditory  (simulator)  73% 

Haptic  (simulator)  40% 

In  table  2  the  ‘preferred  syst< —  -  c. -  .  ^  . 

experienced  the  systems  (the  experimental  group).  The  feedback  display  is  clearly 
favourite,  followed  by  pedal  counterforce. 


0% 

21% 

54% 

0% 

82% 

0% 

0% 

27% 

36% 

10% 

20% 

50% 

is  given  in 

order  of  preference 

of  the  group  who  had 

Table  2.  Preferred  feedback  system 


Control 

Experimental 

Display 

42% 

42% 

Pedal  counterforce 

8% 

17% 

Voice  +  pedal 

0% 

17% 

Voice  +  display 

33% 

8% 

Voice 

8% 

8% 

Display  +  pedal 

8% 

8% 

Discussion 

In  fact  this  is  the  third  experiment  on 

the  road  in  which  feedback  in  one  way  or  the  other 

was  given  to  subjects  about  violations  of  traffic  rules,  in  this  case  primarily  speed. 
Similar  to  the  first  two  experiments  (De  Waard  et  al.,  1994,  De  Waard  and  Brooldiuis, 
1995a),  effects  of  feedback  are  present  again,  but  not  as  large  as  in  previous  experiments 
and  slightly  different.  Specifically  the  effect  on  number  of  speed  violations  was  much 
lower  in  the  present  study.  The  main  difference,  and  probably  largely  explaining  the 
lower  effects,  in  driving  condition  for  the  subjects  this  time  was  a  continuous  feedback 
on  (posted)  speed  limits  and  rule  compliance,  as  opposed  to  incidental  warnings  in  case 
of  violations  in  the  first  two  studies,  A  clear  reduction  in  speed  in  the  experimental 
condition  in  those  cases  where  subjects  clearly  tended  to  violate  the  limit,  i.e.  in  the  50 
km/h  zones,  was  found.  Furthermore,  from  the  data  on  percentage  of  time  driving  above 
limit  (see  figure  1)  it  is  clear  that,  at  least  some  subjects  used  the  continuous  feedback  to 
keep  their  speed  in  the  margin  of  ‘limit  to  limit4-10%’  (amber). 

From  the  acceptance  data  (see  tables  1  and  2)  it  follows  that  acceptance  very  much 
depends  upon  feedback  system,  continuous  feedback  was  accepted  best  of  all  means  of 
feedback  by  far.  The  ratings  for  the  continuous  visual  feedback  were  unusually  high  and 
can  maybe  even  considered  as  (highly)  appreciated.  The  system  that  is  most  proximate  to 
an  actual  speed-restrictor,  haptic  feedback,  is  according  to  the  subjects,  surprisingly 
enough  not  the  most  influential  on  behaviour.  Differences  between  the  two  test 
environments  (simulator  <==>  car)  make  a  comparison  of  effects  on  actual  behaviour  of 
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haptic  versus  visual  display  feedback  difficult  (coufounded). 

An  new,  unexpected  effect  of  the  compound  feedback  was  found,  a  significant  reduction 
in  speed  variation.  One  of  the  reasons  is  the  earlier  mentioned  use  of  the  amber  to  stay  in 
the  margin  of  ‘limit  to  limit+10%’.  The  implication  of  this  finding  is  that  less  variation 
in  driving  speed  could  help  to  harmonize  traffic,  which  is  one  of  the  candidate  tools  to 
reduce  the  number  of  accidents  (see  also  Brookhuis  and  Brown,  1992). 

No  effects  on  workload  were  found  in  this  study,  again  contrary  to  the  first  two 
experiments  as  mentioned.  However,  in  the  latter  studies  the  (slight)  effects  were 
marginally  significant,  while  in  the  present  data  the  (slight)  effects  demonstrated  in  either 
of  the  two  measures  of  mental  load  did  not  attain  significance.  The  implication  of  these 
findings,  in  line  with  Verwey,  Brookhuis  and  Janssen  (1996),  is  that  before 
implementing  telematics  systems,  in  principle  workload  effects  should  be  measured,  just 
to  be  sure,  but  the  type  of  systems  tested  so  far  are  not  implying  alarming  effects. 

Finally,  the  conclusions  of  this  study  can  be  summarized  succinctly  in  three  statements: 

Giving  feedback  about  speed  limit  compliance/violation  is  effective 

Giving  continuous  including  compliance  feedback  is  effective  and  appreciated 

Giving  continuous  including  compliance  feedback  could  help  harmonizing  traffic 
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Abstract 

Motorway  networks  are  important  facilities  to  handle  high  demand  of  traffic  flow  and 
transportation.  In  overloaded  situations  or  in  cases  of  incidents  efficient,  operative 
control  means  are  necessary  to  save  the  operability  of  the  motorway  network.  The  most 
effective  control  means  are  ramp  metering  at  the  entries  of  the  motorway  network, 
variable  speed  limitation  on  individual  sections  of  the  motorway  and  diversion  of  traffic 
flow  using  variable  message  signs. 

In  this  paper  a  macroscopic  simulation  model  for  traffic  flow  is  used  to  find  optimal 
control  strategies  for  traffic  responsive  control  means.  First  the  traffic  model  allows  a 
forecast  of  the  traffic  evalution  on  the  motorway  network.  Further  it  is  used  for  an 
offline-optimization  of  the  control  means  in  certain  traffic  load  situations.  Therefore  an 
optimization  problem  for  the  control  variables  is  formulated  with  a  suitable  performance 
criterion  for  the  traffic  flow.  The  ojfline  optimization  results  are  compared  with  the 
developed  online  traffic  responsive  control  law  in  some  simulation  studies. 

Introduction 

Congestion  on  German  motorway  networks  have  become  a  common  phenomenon 
especially  within  the  holiday  period.  It  leads  to  considerable  delays,  reduced  traffic 
security,  increased  fuel  consumption  and  air  pollution. 

Traffic  control  systems  are  important  facilities  to  preserve  the  capacity  of  the  motorway 
network  and  to  avoid  congestion.  They  require  intelligent  traffic  responsive  control 
strategies. 

In  this  paper  a  traffic  responsive  control  concept  in  motorway  networks  is  presented.  As 
control  inputs  we  take  into  consideraton: 

•  ramp  metering  at  the  on-ramps  of  the  network  by  traffic  signalization, 

•  variable  speed  limitation  imposed  on  individual  sections  of  the  motorway, 

•  traffic  flow  diversion  to  alternative  routes  by  variable  message  signs. 


The  concept  of  a  traffic  responsive  closed  loop  control  is  illustrated  in  Fig.  1 . 

Actual  traffic  data  are  collected  fi*om  appropiate  locations.  These  sample  data  are 
extended  by  estimation  techniques  (  e.g,  as  proposed  by  Cremer,  1976  ).  The  values  of 
the  state  variables  are  then  mapped  by  a  short  control  algorithm  into  the  actual  control 
inputs  which  respond  best  to  the  given  traffic  load. 
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Figure  I:  Structure  of  a  feedback  control  system  for  a  small  network 


This  paper  is  organized  as  follows:  In  the  following  chapter  a  dynamic,  macroscopic 
model  for  the  traffic  flow  in  a  motorway  network  is  introduced.  In  chapter  3,  an  optimal 
control  problem  for  open  loop  control  of  traffic  flow  is  formulated.  This  nonlinear,  large 
scale  optimization  problem  is  solved  by  particular  heuristic  optimization  algorithms. 
Using  tese  results  a  closed  loop  control  law  is  designed  in  chapter  4.  The  efficiency  of 
the  control  law  is  underlined  in  chapter  5  by  a  simulation  of  a  critical  case.  Finally  a 
summary  with  some  conclusions  is  given. 

The  simulation  model  of  traffic  flow 

To  simulate  traffic  flow  in  a  large  network  a  macroscopic,  time  discrete  model  is  used 
which  was  first  introduced  by  Payne  (1971)  and  later  improved  by  Cremer  (1979). 

To  present  the  model  equations  briefly,  let  us  consider  a  section  of  a  motorway,  which  is 
spatially  divided  into  subsections  of  a  length  between  400  and  600  meters. 

. - A, - - - A, - .  - - A.-1 - ^ " 


Figure  2:  Section  of  a  multilane  motorway  with  subsections  and  model  variables 
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Let  T  denote  the  time  step  width.  Then  for  each  subsection  j  ( j-1,  n  )  the  following 
variables  of  the  time  discrete  model  are  introduced: 

Cj  (k)  trajfic  density  in  subsection  j  at  time  kT  [veh/km] 

Vj(k)  mean  speed  of  the  vehicles  within  subsection  j  at  time  kT  [km/h] 

q^(k)  trajfic  volume  from  subsection  j  into  subsection  j+1  during  the  time 

interval  kT  <  t  <  (k  +  1)T  [veh/h] 

qr  (k ),  (k )  entering  or  leaving  ramp  volumes  during  kT  <  t  <  (k  +  1)T  [veh/h] 

Now  we  can  formulate  difference  equations  for  the  state  variables  Cj(k)  and  Vj(k): 


Cj(k  + 1>  =  Cj  (k)  +  (k)  -qj(k)  +  5^jq,(k)  -  KjqJk)] 


(1) 


Vj(k  +  l)  =  Vjik)+^[v{cj(k),U:,(k))-v^(k)]+f-[vj(k){vj_i(k)-Vj(k))'\ 


V  T 

Cj^i(k)-Cj(k) 

X  Ay 

Cj(k)+K 

(2) 


Equation  (1)  is  a  simple  balance  of  vehicles  and  equation  (2)  was  derived  from  empirical 
considerations.  The  first  term  in  brackets  on  the  right  hand  side  describes  a  retarded 
adaptation  of  the  average  speed  to  the  stationary  speed-density  characteristic  V(c,U2). 
The  second  term  represents  convection  of  spatial  speed  profile  in  the  downstream 
direction  and  the  third  term  models  drivers  anticipation  of  a  density  variation  as  seen  in 
downstream  direction. 

For  the  volumes  between  the  subsections  the  following  approximate  expression  taken 
from  hydrodynamical  relations  is  meaningful: 

qj(k)  =  acj(k)vj(k)  +  ('l-a)  Cj^i(k)Vj^,(k)  (3) 

The  model  parameters  cx,  k,  x,  v  and  the  speed-density-characteristic  (  see  below )  have  to 
be  calibrated  by  real  measurements.  For  German  motorways  the  calibration  was 
performed  in  Cremer  and  Papageorgiou  (1981).  For  the  time  step  T  a  value  of  10  sec  has 
shown  to  be  a  suitable  value.  Care  has  to  be  taken  that  the  maximum  speed  does  not 
exeed  the  ratio  Aj/T . 


The  influence  of  speed  limitation  is  modelled  by  the  control  variable  U2 .  Based  on 
investigations  of  the  effect  of  speed  limitations  on  German  motorways  (Zackor,  1972) 
the  following  relationship  for  a  stationary  speed-density  characteristic  has  been 
formulated: 
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V(C,U2)~VfU2'  1- 


C 


1(3-2u2) 


(4) 


where  is  the  average  free  flow  velocity,  is  the  maximum  density  at  full  congestion 
and  1  and  m  are  positive  real  numbers.  The  control  variable  M2  has  the  following 
correspondences: 


=  1: 

unlimited  free  speed 

=  0.82 : 

speed  limitation  to  100  km/h 

=  0.65 : 

speed  limitation  to  80  km/h 

=  0.5: 

speed  limitation  to  60  km/h 

Figure  3:  Speed-density  characteristic  and  volume-density  characteristic  for  different 
speed  limitations 


Ramp  metering  by  control  input  m^  is  modelled  by  the  following  equation: 
qr  'f(c^(k)) 

q^(k)  =  min'  q^ -u^ik)  (5) 

^d(k)  +  qjk) 

Thus,  the  entering  volume  is  either  limited  by  congestion  on  the  motorway  (  function 
f(ci))  or  by  the  green  time  quota  u^  of  the  ramp  metering  signal  or  by  the  number  of 
waiting  ( q^ )  and  arriving  {q^)  vehicles. 

To  model  the  leaving  ramp  volumes  at  intersections  or  off-ramp  some  knowledge  about 
the  origin  destination  pattern  is  necessary.  For  simplifying  reasons  we  assume  constant 
local  OD-quotas  (0  <  e  <  1) : 
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(6) 


q^jk)  =  (l-e)^qjk) 

If  diversion  of  traffic  flow  due  to  congestion  or  an  incident  on  the  main  route  is 
recommended  by  the  control  variable  e  {0,1}  >  the  off-ramp  volume  is  increased  by  an 

additional  trafficstream  qjk): 
qJk)  =  u^(k)‘E^^(\-e)  qJk) 

q"jk)^(l-E)-qJk)-qJk) 

denotes  the  divertable  portion  of  the  traffic  flow.  The  diverted  traffic  flow  has  to  be 
balanced  separately  on  the  alternative  route  in  order  to  secure  correct  origin  destination 
flow.  For  this  reason  density  and  volume  have  to  be  calculated  separately  for  the  diverted 
flow  in  Eq.  (1)  and  (3),  The  average  speed  (Eq.  (2))  has  to  be  calculated  by  taking  the 
total  density. 

The  control  variables  Ui,U2  and  1/3  are  defined  at  several  places  in  the  network  and  can 
be  changed  after  a  period  of  seconds  which  should  be  chosen  not  too  small  for  reason 
of  acceptance.  We  have  chosen  a  value  of  =  5  min. 

Optimal  control  problem 

In  order  to  determine  optimal  controls  for  a  given  traffic  demand  pattern  we  need  a 
suitable  performance  criterion  by  which  the  impact  of  the  control  variables  on  the  traffic 
flow  can  be  quantified.  Several  objective  functions  may  be  found  in  literature  (e.g.  in 
Isaksen  and  Payne  (  1973)  or  Cremer  and  Heischmann  (1987)).  A  meaningful  objective 
is  to  minimize  overall  travel  time  within  the  network  during  the  time  period  under 
consideration.  The  only  disadvantage  is  that  the  effects  of  the  control  actions  can  reach 
beyond  the  chosen  period  and  are  therefore  not  accounted  properly. 

The  total  delay  time  in  the  network  is  closely  related  to  the  overall  travel  time  and  avoids 
the  mentioned  disadvantage  (Cremer  and  Schoof  (1989)).  Total  delay  time  is  calculated 
as  a  function  of  the  model  variables  by  taking  a  sum  of  three  different  terms: 

•  delay  time  from  the  waiting  queues  on  the  i-th  on-ramp: 

I\{.k)  =  TUl{k)  (8) 

•  momentary  delay  time  in  subsection  j  of  the  motorway: 

li(k)  =  AjCj{k)T  (9) 

•  delay  time  caused  by  driving  on  the  deviation  route: 

L,(k)=Tq,(k)^^  (10) 

where  /„  and  4  are  the  lenghts  of  the  main  route  and  the  alternative  route. 
Total  delay  time  is  then  the  sum  over  time  and  space  of  the  momentary  terms: 
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P  =  Z  Zl^(k)  +  tli{k)  +  L,{k) 

k:=l  L  1=1  >1 

With  this  performance  criterion  we  can  now  formulate  an  optimization  problem: 

Given  a  time  discrete  model  for  the  traffic  flow  through  a  motorway  network  as  nonlinear 
differential  equation  system: 

x(k^-\)  =  x(k)  +  f{x(kX  u(k),  qo(k))  (12) 

with  known  initial  state  x(0)  and  known  entry  flows  qo(k),  k  =  X...,K, 

Find  sequences  for  all  control  variables  |mJ (k), u{(k),  u^(kjj over  a  chosen  time  period  K 
which  minimize  the  total  delay  time  P  (1 1)  under  the  following  constraints: 

0<ui(k)<l 

0.5<ui(k)<l  (13) 

u^(k)G  {0,1} 

Values  can  be  changed  after  a  period  of  r„  seconds  ( e.g.  5  min, ), 

This  problem  can  be  classified  as  a  large  scale,  nonlinear,  mixed  integer  optimization 
problem.  To  solve  such  a  complex  problem,  several  optimization  techniques  have  been 
tested.  The  best  results  were  achieved  by  the  method  of  „Threshold  Accepting**,  which 
was  developed  by  Dueck  and  Scheuer  (1990). 

In  one  optimization  step  a  complete  simulation  of  the  traffic  evolution  in  the  network  is 
necessary,  so  the  performance  of  the  simulation  model  is  very  important. 

Traffic  responsive  control  law 

The  optimization  procedure  is  only  suitable  for  an  offline-control  of  the  traffic  flow, 
because  of  the  complexity  and  the  computational  effort. 

For  an  online-control- system  as  shown  in  Figure  1  a  simple  control  law  is  needed,  which 
maps  the  actual  state  variables  into  new  control  decisions.  From  typical  case  studies  and 
analysis  of  the  static  traffic  control  problem  a  new  control  law  for  the  online  traffic 
control  problem  in  urban  traffic  corridors  was  derived  ( Schoof  (1991) ). 

We  present  the  control  law  briefly  adapted  for  motorway  networks: 

7.  Determination  of  the  on-ramp  control  values  u\(k)  for  the  i-th  section 

In  stationary  state  of  traffic  flow  the  ramp  metering  control  variables  can  be  calculated  by 
a  simple  linear  optimization  scheme,  hi  case  of  congestion  a  special  algorithm  is 
necessary  to  reduce  the  congestion. 

In  the  i-th  section  the  following  constraint  has  to  be  fulfilled  (  see  Fig.  4): 
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(14) 


The  incoming  static  traffic  volumes  are  given  at  the  network  boundarys  or  can  be 
calculated  successively. 


congestion 


Figure  4:  Necessary  constraint  for  congestion  reduction 


From  this  the  values  u\  can  be  determined  as  follows: 

f  Qmp  congestion  (e.g.  =  4000  veh  /  h  for  2  lanes) 

~  I  min  \q) }  in  case  of  congestion  (  max  {c}  }  >  ) 

0  for9-^-r‘<0 

“I  =  {(log  « otherwise 

1  toxq^^-r-'^qr 

If  the  reduction  of  the  incoming  traffic  volume  is  not  sufficient,  other  upstream  onramps 
have  to  be  closed. 

2.  Determination  of  the  speed  limitation  values  u\(k)  for  the  i-th  section 

The  influence  of  speed  limitation  to  the  stationary  speed-density  characteristic  V(c,M2) 

is  formulated  in  Eq.  (4).  With  Q(c,U2)-c^V(c,U2)  and  ^~(c,U2)  =  0  the  optimal 

speed  limitation  for  a  stationary  density  ccan  be  calculated.  The  functional 
correspondence  u^  =  U2(c)is  depicted  in  Fig.  5. 

To  obtain  a  characteristic  density  value  in  the  section,  a  weighted  average  density  c '  is 
calculated  with  heigher  weight  to  the  heigher  density  values: 


‘  =YiWj-c)(k),  X)w';=l  and  U2=U2{c') 


If  congestion  is  detected  (  max{c}  }>  c^^it)  >  speed  limitation  of  W2  =  0.65  (  80  km/h )  is 
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recommended.  If  other  upstream  onramps  must  be  closed,  speed  limitaion  in  the 
upstream  sections  are  set  to  80  km/h  respectively. 


Figure  5:  Stationary  control  characteric  U2(c) 


3.  Determination  of  the  route  diversion  decision  u^(k) 

For  calculation  of  the  route  diversion  decision  it  is  necessary,  to  estimate  the  influence  of 
the  decision  to  the  performance  on  the  main  route  and  on  the  alternative  route.  In 
stationaiy  state  of  traffic  flow  the  delay  time  in  the  i-th  section  can  be  determined  by: 


and  /j-  denotes  the  length  of  the  i-th  section. 


The  delay  time  caused  by  driving  on  the  deviation  route  is  in  this  case: 


L,(qJ^KTq^ 


If  ^ <  ql^for  all  sections  i,  recommendation  of  the  route  diversion  can  be  determined 
by  calculating: 

A,  =  (i9) 

16/, „  16/^ 

If  Aj  >  0,  set  u^(k)  =  !.(/„  denotes  the  set  of  sections  on  the  main  route,  4  of 

sections  on  the  alternative  route  respectively ). 

The  revocation  of  the  route  diversion  can  be  determined  by  calculating: 


If  A2  >  0,  set  uJk)  =  0. 


If  q'  >^^^^for  a  section  i,  the  route  diversion  is  determined  with  the  aim  to  avoid 
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congestion.  For  a  detailed  algorithm  see  Schoof  (1991). 

Results 

To  demonstrate  the  efficieny  of  the  traffic  responsive  control  law,  a  topology  like  Fig.l 
was  chosen.  The  length  of  the  main  route  is  5  km  to  node  1,  then  15  km  to  node  2  and 
another  5  km.  The  length  of  the  alternative  route  between  node  1  and  2  is  20  km. 

The  simulation  period  is  1  hour,  and  on  the  main  route  an  incident  occurs  from  min.  17 
to  33  by  which  one  of  two  lanes  is  blocked. 

Fig.  6-8  show  the  evaluation  of  the  density  at  the  main  route  in  case  of  no  control,  traffic 
responsive  control  and  optimal  control.  In  case  2  the  route  diversion  is  recommended 
from  min,  20  to  40,  in  case  3  from  min.  15  to  30. 


t  [min} 


Figure  6:  Time  space  diagram  of  density  on  the  main  route  without  control 


Figure  7:  Time  space  diagram  of  density  on  the  main  route  with  traffic  responsive 
control  law 
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Figure  8:  Time  space  diagram  of  density  on  the  main  route  with  optimal  control 


The  performance  index  for  the  overall  network  in  the  three  cases  is  shown  in  the 
following  table: 


performance  index  [veh  h] 

no  control 

396.2 

traffic  responsive  control 

303,5 

optimal  control 

285,4 

Conclusions 

In  this  paper  a  traffic  responsive  control  law  is  presented  to  determine  optimal  control 
sequences  for  traffic  flow  in  a  motorway  network.  The  control  means  are  ramp  metering, 
speed  limitation  and  route  diversion. 

First  a  dynamic  macroscopic  model  for  the  traffic  flow  in  a  motorway  network  as  a 
simulation  tool  is  introduced.  Second  an  optimal  control  problem  was  formulated.  To 
solve  this  large  scale,  nonlinear,  mixed  integer  optimization  problem  special  optimization 
techniques,  e.g.  the  method  of  „Threshold  Accepting^,  are  necessary. 

For  an  online-control-system  a  traffic  responsive  control  law  is  necessary,  which  was 
derived  by  investigation  of  the  stationary  state  of  traffic  flow.  The  efficieny  of  the  control 
concept  is  demonstrated  by  the  results  of  a  case  study. 

Future  work  is  directed  to  extend  the  control  law  for  multiple  alternative  routes  with 
conflicting  requirements.  Other  research  tasks  are  a  sensitivity  analysis  or  the  reduction 
of  the  measured  information. 
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Abstract 

The  possible  errors  drivers  can  make  interpreting  different  traffic  signal/sign 
combinations  at  signalized  intersections  fall  into  two  error  categories:  False  yield  and 
false  go  responses.  In  order  to  understand  better  the  possible  reasons  for  false  yield  and 
false  go  responses,  younger  and  older  drivers  had  to  interpret  different  signal/sign 
combinations  while  operating  a  driving  simulator.  The  combinations  of  traffic  signals 
and  signs  were  selected  such  that  the  elements  in  those  combinations  either  conveyed  a 
yield  response,  a  go  response,  or  both,  to  different  degrees.  The  percentage  of  errors 
drivers  made  was  the  dependent  variable.  We  predicted  that  the  percentage  of  errors 
would  vary  systematically  with  the  compatibility  of  the  messages  implied  by  the  sign  and 
signal.  The  predictions  were  consistent  with  the  observed  errors  in  the  majority  of 
combinations.  Thus,  we  are  able  to  predict  which  particular  signs  will  be  most  effective 
with  a  given  signal.  Theoretical  and  practical  gains  clearly  follow  from  this  capability. 

Introduction 

Drivers  have  to  react  to  different  signals,  one  of  which  is  the  traffic  signal  at 
intersections.  Oftentimes  the  design  and  location  of  these  traffic  signals  does  not  allow 
their  efficient  interpretation  by  drivers  (Bonneson  and  McCoy,  1994;  Williams, 
Ardekani,  and  Asante,  1992).  This  leads  to  errors,  where  drivers  sometimes  go  when 
they  should  yield  (false  go),  and  sometimes  yield  when  they  should  go  (false  yield), 
Williams  et  al.  (1992),  for  instance,  conducted  a  mail  survey,  in  which  different 
pictographic  designs  of  signalized  intersections  were  sent  to  hundreds  of  drivers  in  Texas. 
Drivers  had  to  decide  whether  a  left  mm  was  allowed  during  the  presented  signal  interval, 
and  if  so,  whether  it  was  permitted  or  protected.  Based  on  a  sample  of  close  to  900 
responses,  the  authors  concluded  that  the  simultaneous  presentation  of  a  red  globe  and  a 
green  arrow  on  a  horizontal  five  section  head  should  be  avoided  since  this  combination 
produced  a  great  many  false  yield  decisions.  False  yield  responses  were  also  found  in  a 
similar  survey  conducted  by  Bonneson  and  McCoy  (1994),  In  their  survey,  the  authors 
tested  how  well  motorists  understood  whether  a  signal  design  indicated  either  a  protected 
left  mm  (green  arrow  and  red  globe),  a  permitted  left  turn  (green  globe,  i.e„  drivers  can 
mm  left  if  there  is  no  oncoming  traffic),  or  an  overlap  condition  (i.e.,  left  mrn  green 
arrow  for  left  mm  indication  and  green  globe  for  through  indication).  They  found  that 
the  overlap  indication  was  least  understood  (i.e.,  false  yield  responses  for  about  half  of 
the  respondents),  and  permitted  and  protected  indications  were  understood  equally  well. 
Regarding  sign  use,  a  significantly  higher  correct  response  rate  occurred  without  the  use 
of  signs.  The  sign  used  was  “left  mm  yield  on  green  (symbolic  green  globe)”.  It 
improved  driver  understanding  only  in  the  permitted  indication  (green  globe)  whereas 
during  overlap  and  protected  indication  it  appeared  to  confuse  drivers,  a  result  also 
consistent  with  other  smdies  (Hummer,  Montgomery,  and  Sinha,  1990;  see  also  Williams 
et  al.,  1992), 

In  addition  to  false  yield  responses,  false  go  responses  have  also  been  observed.  For 
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example,  Staplin  and  Fisk  (1991)  ran  an  experiment  in  which  subjects  were  shown  a 
series  of  still  frames  representing  the  approach  to  an  intersection.  Cars  were  positioned  in 
the  opposite  lane  of  the  intersection.  Upon  the  onset  of  an  auditory  signal  subjects  had  to 
press  either  the  brake  or  accelerator  pedal  to  indicate  either  a  no  go  or  go  decision  as  a 
response  to  the  depicted  traffic  signal  and  sign.  Staplin  and  Fisk  found  that  if  the  signal 
was  a  green  globe  the  percentage  of  false  go  responses  ranged  from  13.1%  to  30.4%  for 
younger  adults,  and  from  44%  to  64%  for  older  adults  depending  on  the  accompanying 
sign.  For  example,  if  a  green  globe  was  presented  with  the  sign  “left  turn  yield  on  green 
(symbolic  green  globe)”,  13.1%  of  younger,  but  44%  of  older  adults  indicated  that  it  is 
safe  to  mm  left.  If  the  signal  was  a  green  globe  and  the  sign  was  “protected  left  on  green 
arrow”  30.4%  of  younger,  but  64%  of  older  adults  made  a  false  go  response. 

The  reviewed  results  indicate  that  the  presence  of  an  auxiliary  sign  can  help  as  well  as 
impair  driver  understanding.  Furthermore,  it  appears  that  understanding  of  the  traffic 
signal  and  sign  depends  on  whether  both  elements  convey  the  same  message.  For 
example,  in  Staplin  and  Fisk's  (1991)  experiment,  more  errors  were  made  when  the  green 
globe  was  accompanied  by  the  sign  “protected  left  on  green  arrow”,  than  when  the  sign 
was  “left  mm  yield  on  green  (symbolic  green  globe)”.  Since  the  latter  sign  and  signal 
both  imply  that  the  driver  should  yield  if  making  a  left  mm,  whereas  the  former  sign  and 
signal  imply  different  actions  (go  and  yield,  respectively),  drivers  may  respond  correctly 
more  often  to  the  combination  for  green  globe  and  “left  mm  yield  on  green  (symbolic 
green  globe)”  sign  than  they  do  to  the  combination  of  a  green  globe  and  “protected  left  on 
green  arrow”  sign. 

In  an  attempt  to  investigate  the  possible  reasons  for  incorrect  responses  as  well  as  to 
replicate  the  results  of  the  previous  smdies  we  tested  drivers  under  realistic  driving 
conditions  on  a  driving  simulator.  Drivers  approached  a  4-way  signalized  intersection 
and  had  to  decide  whether  it  is  safe  to  mm  left  in  the  presence  of  oncoming  traffic  in  the 
opposite  lane  of  the  given  intersection.  We  expected  that,  in  general,  fewer  errors  would 
be  made  in  our  smdy  compared  to  the  reviewed  smdies  because  subjects  could  interact 
with  and  take  into  account  approaching  traffic  when  making  a  response.  We  further 
theorized  that  errors  would  vary  with  the  degree  of  compatibility  between  the  conveyed 
messages  of  the  traffic  signal  and  the  sign.  Thus,  in  conditions  in  which  a  protected  left 
mm  stams  is  conveyed  by  the  signal  (i.e.,  a  green  arrow  with  or  without  a  green  globe),  a 
compatible  sign  that  conveys  verbally  the  same  stams  [e.g.,  “protected  left  on  green 
arrow”]  should  allow  better  understanding  than  an  incompatible  sign  that  conveys  a 
“yield”  message  [e.g.,  “left  turn  yield  on  green  (symbolic  green  globe)”].  The  possible 
errors  drivers  can  make  for  those  signal/sign  combinations  implying  a  protected  left  turn 
stams  are  false  yield  responses.  Conversely,  in  conditions  in  which  a  permitted  left  turn 
stams  is  conveyed  by  the  signal  (i.e.,  green  globe),  a  compatible  sign  that  conveys 
verbally  the  same  stams  [e.g.,  “left  mm  yield  on  green  (symbolic  green  globe)”]  should 
allow  better  understanding  than  an  incompatible  sign  that  conveys  a  “protected  left  turn” 
message.  The  possible  errors  drivers  can  make  interpreting  those  signals  which  imply  a 
permitted  left  mm  stams  are  false  go  responses.  For  combinations  with  a  red  globe  or 
arrow  no  predictions  can  be  made  with  respect  to  the  compatibility  between  signal  and 
sign,  since  no  sign  used  referred  to  a  red  arrow  or  globe.  However,  we  would  expect  that 
a  red  light  indicates  a  yield  or  stop  response  and  thus  should  benefit  from  a  sign  that 
implies  a  yield  response. 
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Method 

Subjects.  32  drivers  were  tested;  16  of  which  were  between  20  to  32  years  of  age 
(average  23)  and  had  at  least  four  years  of  driving  experience.  The  other  16  drivers  were 
between  the  age  of  63  and  84  (average  71),  Subjects  were  reimbursed  with  $10  for  each 
session  in  which  they  participated. 

Subjects  were  tested  on  the  two  cognitive  tests  of  the  Wechsler  Adult  Intelligence  Scale 
(1955),  the  digit  span  test  and  the  vocabulary  test.  On  average,  older  adults  had  a 
vocabulary  score  of  38.25  and  a  digit  span  score  of  1 1.31;  younger  adults  had  an  average 
vocabulary  score  of  37.38  and  a  digit  span  score  of  12,31.  These  tests  were  used  to 
ensure  that  both  age  groups  had  equivalent  educational  backgrounds.  Older  and  younger 
adults  did  not  differ  significantly  in  both  test  scores  as  established  by  a  tow-tailed  t-test  at 
a  =  0.05  (r  =  1.069  for  the  vocabulary  test,  and  r  =  1.54  for  the  digit  span  test). 

Apparatus.  The  driving  simulator  consisted  of  a  sedan  (Saturn)  positioned  in  front  of  a 
screen.  A  virtual  driving  scenario  was  projected  onto  the  screen  using  a  Sony  Multiscan 
Projector  VPH-1272Q/1272QM,  1993©.  The  car  was  equipped  with  the  usual  devices 
installed  in  a  car  with  an  automatic  gear  shift.  Steering  wheel,  brake  and  accelerator 
pedals,  as  well  as  the  gear  shifting  stick,  allowed  realistic  driving  maneuvers.  The 
scenarios  were  designed  using  a  real-time  3D  modeling  environment  (Designer’s 
Workbench  3.1)  developed  by  Coryphaeus  Software,  Inc,  1995©.  Driving  within  a  built 
scenario  and  interaction  with  other  cars  were  accomplished  using  another  software 
package  developed  by  Monterey  Technologies,  Inc.  1995©. 

Visual  Database.  One  scenario  consisted  of  four  4-way  signalized  intersections  which 
were  built  continuously,  one  intersection  after  the  other.  There  were  two  through  lanes 
and  one  separate  left-turn  lane  for  the  driver  with  a  white  stop  line  at  the  end  of  this  lane. 
The  length  of  the  left  turn  lane  was  about  33.5  meters.  The  intersections  were  built 
according  to  MUTCD  (Manual  on  Uniform  Traffic  Control  Devices,  1988,  4b-12/4b-13) 
recommendations.  One  intersection  was  31  meters  across  and  31  meters  wide.  The 
traffic  signal  was  visible  within  a  40  degree  visual  angle  from  the  position  of  the  driver  at 
the  white  stop  line  at  a  distance  of  14.5  meters  to  the  traffic  light.  The  height  of  the  lower 
part  of  the  casing  of  the  traffic  signal  was  4.9  meters  with  the  sign  positioned  to  its  right. 

The  relevant  traffic  signals  consisted  of  either  a  2x3  signal  head-  (Signals  4,  6,  and  8)  or  a 
1x5  signal  head  (Signals  1,  2,  3,  5,  and  7)  arrangement  as  depicted  in  Figure  1.  The  size 
of  the  signal  and  signs  differed  from  MUTCD  recommendations  in  that  they  were  bigger 
than  real  signs  to  allow  for  better  legibility.  Signs  were  legible  at  a  distance  of  about  25 
meters.  The  signals  1  to  8  were  combined  with  each  of  the  signs  A  to  D  depicted  in 
Figure  1  resulting  in  32  relevant  combinations.  Not  shown  are  eight  additional 
signal/sign  combinations  that  were  chosen  as  “fillers”  to  induce  variation  in  the  scenarios 
and  to  convey  the  impression  of  an  “realistic  driving”  task,  resulting  in  a  total  of  40 
signalized  intersections.  “Fillers”  included  two  additional  signs,  i.e.,  “left  on  arrow  only” 
or  “left  turn  not  protected”.  All  signs  are  currently  in  use  in  the  United  States.  Signals  in 
combination  with  signs  D  and  B  allowed  us  to  compare  directly  the  effects  of 
compatibility  between  the  signals  and  the  sign.  Sign  A  (no  sign)  represents  a  control 
condition  in  which  only  the  effects  of  the  shown  signal  are  tested,  whereas  sign  C,  “left 
lane  must  turn  left”,  allows  us  to  assess  the  distracting  effects  of  signs  that  do  not  relate  to 
driver  actions  with  respect  to  the  depicted  signal. 
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Figure  1:  Signal  (1-8)  and  signs  (A,  B,  C,  D).  Each  of  the  eight  signals  is  combined  with 
each  of  the  signs.  The  dimensions  are  not  drawn  to  scale. 

Scenario  Dynamics.  Some  of  the  signals  included  a  red  light  (red  globe  and/or  red  arrow) 
which  induced  the  driver  to  stop  at  the  intersection.  If  the  driver  slowed  down  and 
continued  to  slowly  approach  the  intersection  the  red  light  would  change  to  a  green  light, 
i.e.,  a  green  arrow  and/or  a  green  globe.  When  the  driver  approached  the  intersection 
there  was  approaching  traffic  (2  to  4  cars)  in  the  opposite  lane  for  through  traffic.  The 
velocity  of  the  approaching  cars  was  set  around  100%  relative  to  that  of  the  driver. 
Movement  onset  of  the  approaching  cars  was  triggered  at  a  distance  of  about  100  to  300 
meters  to  the  driver.  At  a  certain  distance  (between  40  and  80  meters)  from  the 
intersection  the  approaching  cars  drove  independently  of  the  velocity  of  the  driver  and 
continued  through  the  intersection  or  stopped  at  the  traffic  light  depending  on  the  traffic 
signal  shown  to  the  driver.  If  the  driver  was  shown  a  protected  left  turn  status  (signal- 
sign  combinations  2A-D,  5A-D,  7A-D)  the  approaching  cars  stopped  at  the  intersection. 
If  the  driver  saw  a  permitted  left-turn  status  displayed  by  the  traffic  signal  and  sign 
(combinations  lA-D)  the  approaching  cars  proceeded  to  drive  through  the  intersection.  If 
the  driver  had  to  come  to  a  stop  because  of  a  red  light  (red  globe  and/or  red  arrow)  the 
approaching  traffic  stopped  also.  After  the  signal  changed  and  the  driver  could  continue, 
the  approaching  cars  proceeded  accordingly,  i.e.,  dependent  on  the  traffic  signal  status 
shown  to  the  driver. 
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Design.  Within  a  scenario  the  four  intersections  were  arranged  such  that  no  signal  or  sign 
occurred  twice  in  a  row.  Since  there  were  40  traffic  signal  and  sign  combinations,  10 
scenarios  were  constructed.  The  order  of  the  10  scenarios  presented  to  each  subject 
within  an  age  group  was  different. 

Procedure.  Before  the  experiment  drivers  were  given  instructions  and  familiarized  with 
the  driving  simulator  in  two  practice  sessions.  In  the  first  practice  session  the  driver  sat 
on  the  passenger  seat  and  the  experimenter  drove  through  one  scenario  to  show  the  driver 
the  dynamics  of  the  car  and  to  demonstrate  steering  and  braking  behavior.  After  that,  the 
driver  drove  through  one  scenario  in  a  second  practice  session.  The  practice  scenarios 
consisted  of  four  signalized  intersections,  however,  no  signals  or  signs  were  visible.  The 
practice  scenarios  were  otherwise  identical  to  the  experimental  scenarios.  Drivers  were 
instructed  to  drive  between  10  and  15  miles  per  hour  (16  to  24  km/h)  and  to  make  no 
abrupt  steering  or  braking  movements  to  avoid  potential  simulator  discomfort  (e.g., 
dizziness,  nausea).  Controlled  driving  was  emphasized. 

Subsequent  to  the  practice  sessions  the  experiment  began.  The  driver  was  informed  that 
he/she  would  see  a  signalized  intersection.  There  would  be  approaching  traffic  in  the 
opposite  lane  of  the  intersection.  Drivers  were  supposed  to  approach  the  intersection  and 
turn  into  the  left-turn  lane.  Their  task  was  to  turn  left  at  the  intersection  if  they  thought  it 
was  safe  to  do  so  given  the  depicted  traffic  signal.  To  ensure  that  drivers  read  the  sign 
they  were  informed  at  the  beginning  of  the  experiment  that  they  would  be  given  a  sign 
comprehension  test  at  the  end  of  the  study.  If  drivers  decided  it  was  safe  to  turn  left  they 
should  turn  across  the  intersection.  If  they  decided  it  was  not  safe  they  should  stop  at  the 
white  stop  line  or  yield  at  the  intersection,  and  then  continue  the  left  turn  when  permitted 
either  by  changes  in  the  traffic  or  by  changes  in  the  status  of  the  traffic  signal,  finally 
driving  to  the  next  intersection.  Drivers  were  further  instructed  to  indicate  verbally  their 
decision  by  saying,  for  instance,  “I  have  to  yield”  or  “I  can  go”.  Drivers  were  informed 
that  the  other  cars  would  drive  according  to  traffic  regulations.  After  four  intersections, 
the  driver  stopped  and  took  a  break  for  about  2  minutes  during  which  time  the  next 
scenario  was  loaded.  After  completing  a  pair  of  scenarios  the  driver  could  take  a  break  of 
5  minutes  if  he/she  wished  to  do  so.  If  drivers  drove  15  miles  per  hour  (24  km/h)  they 
needed  about  2  minutes  to  drive  through  the  entire  scenario.  The  entire  experiment  lasted 
about  one  hour. 

Dependent  Variables.  A  driver  error  was  recorded  if  the  driver  falsely  stopped  (yielded) 
or  falsely  went  through  the  intersection.  Driver  behavior  (yield  or  stop  vs.  go)  and  errors 
were  recorded  manually.  The  velocity  of  the  driver’s  car  (meters/second)  and  the  x,  y, 
and  z  coordinates  of  the  car  were  recorded  at  a  frequency  of  33  Hertz.  The  velocity  data 
were  analyzed  over  data  points  within  a  2- second  window  before  and  after  subjects 
passed  the  white  stop  line  at  the  intersection  to  establish  the  validity  of  the  manual  coding 
procedure. 

Results 

Three  older  subjects  canceled  the  experiment  due  to  experienced  nausea  and  were 
replaced.  All  drivers  were  able  to  give  a  correct  definition  of  the  used  signs.  A  repeated 
measures  ANOVA  was  computed  over  the  velocity  data  with  driver  behavior  (yield  or 
stop  vs.  go)  and  age  group  (young  vs,  older  adults)  as  factors.  The  velocity  data  are 
shown  in  Table  1.  Depicted  are  the  average  velocities  (in  km/h)  for  correct  responses. 
Subjects  drove  significantly  slower  when  they  correctly  yielded  than  when  they  correctly 
230 


drove  through  the  intersection,  F(l,30)=  91.  28,  p  <  0.001,  and  older  drivers,  in  general, 
drove  slower  than  younger  drivers,  F(l,30)  =  65.32,  p  <  0.001.  The  interaction  between 
age  and  driver  behavior  was  also  significant,  F(l,30)  =  37.29,/?  <  0.001,  indicating  that 
the  difference  between  yield  and  go  responses  was  more  pronounced  for  younger  adults. 


Table  1:  Average  velocity  in  km/h  for  correct  go  and  correct  yield  responses.  Velocity 
data  are  averaged  over  data  points  within  a  2-second  window  before  and  after  drivers 
passed  the  white  stop  line. 


Correct  Yield 

Correct  Go 

Total 

Young 

10.65 

18.66 

14.66 

Old 

9.46 

11,22 

10.34 

AU 

10.05 

14.94 

Pair-wise  comparisons  were  made  between  the  error  percentages  of  selected  signal  and 
sign  combinations.  The  selected  test  statistic  was  z*  =  (pi-p2)/[p(i"p)(l/^+l/w)]  , 
where  pi  and  p2  are  the  respective  proportions  and  m  and  n  the  selected  sample  sizes. 
Parameter  p  is  the  pooled  proportion  value,  p  =  [{ml{m+n))p\+{nl{m+n))p/\.  The 
proportion  differences  were  tested  at  a  =0,01.  Summary  data  are  depicted  in  Table  2. 
The  overall  error  rate  was  3.9%,  There  were  significantly  more  false  yield  than  false  go 
responses,  z*  =  6,99,  There  was  no  age  effect  reflected  by  the  error  data,  z*  =  0.64. 
The  error  data  collected  for  the  different  traffic  signal  and  sign  combinations  are  depicted 
in  Figure  2  for  the  “false  yield”  responses,  and  Figure  3  for  “false  go”  responses.  In 
Figures  2  and  3,  the  signal/sign  combinations  are  depicted  in  a  matrix-like  arrangement. 
The  number  of  the  signal  and  the  letter  of  the  sign,  as  well  as  error  percentages  are 
depicted  in  the  “cells”.  For  signals  displaying  a  protected  left  turn  status  Sign  B  (16.7%) 
produced  significantly  more  “false  yield”  responses  than  sign  D  (4.2%),  z*  =  2.83.  Sign 
C  (13.5%)  produced  significantly  more  false  yield  responses  than  no  sign  (sign  A,  3.2%), 
z*  =  2.61  (see  last  row  of  Figure  2).  No  significant  differences  between  the  signs  were 
obtained  for  “false  go”  responses  (see  last  row  of  Figure  3). 


Table  2:  Error  Percentages  for  false  yield  and  false  go  responses  for  the  different  age 
groups. 


False  Yield 

False 

Total 

Young 

9.89% 

0.94% 

4.3% 

Old 

8.85% 

0.31% 

3.5% 

All 

9.37% 

0.62% 

231 


/r=\ 


LEFT TURN 

YIELD 

ON  GREEN 


PROTECTED 
LEPTON 
GREEN  ARROW 


No  Sign 


2B  12.5% 

old  6.2% 
young  18.8% 


2D  3.1% 

old  6.2% 


2A  0% 


LEFT  LANE 
MUST 
TURN  LEFT 


2C  62% 
young  12.5% 


5B  28.1% 
old  31.2% 


young  25.0% 


5D  6.2% 

old  6.2% 


young  6.2% 


5A  6.2%  SC  25.0% 

old  6.2%  old  25% 

young  6.2%  young  25% 


7B  9.4% 

old  6.2% 
young  12.5% 


7D  3.1% 

young  6.2% 


7A  3.1% 
old  6.2% 


7C  9.4% 

old  12.5% 
young  6.2% 


Error  16.7%  4.2% 

Percentage  old  14.6%  old  4.2% 

per  Sign  young  18.6%  young  4.2% 


3.2%  13.5% 

old  2.1%  old  14.5% 

young  4.2%  young  12.5% 


red 


EZ]  green 


Figure  2:  Percentage  of  false  yield  responses  for  signal  and  sign  combinations  for 
younger  and  older  drivers.  Each  of  the  signals  shown  in  the  left  column  is  paired  with 
each  of  the  signs  in  the  first  row.  The  respective  signal/sign  combinations  and  the 
percentage  of  errors  are  depicted  in  the  cells'". 
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LEFT  TURN 

YIELD 

ON  GREEN 


PROTECTED 
LEFT  ON 
GREEN  ARROW 


No  Sign 


LEFT  LANE 
MUST 
TURN  LEFT 


ID  3.1% 

old  6.2% 


1C  3.1% 


young  6.2% 


3C  3.1% 


young  6.2& 


4A  0%  4C 


6A  0%  6C  3.1% 


young  6.2% 


8B  0% 


8C  0% 


Error 
Percentage 
per  Sign 


0.62% 
old  1.2% 


red  i-y-j  green 


young  3.8% 


Figure  3:  Percentage  of  false  go  responses  for  signal  and  sign  combinations  for  younger 
and  older  adults.  Each  of  the  signals  shown  in  the  left  column  is  paired  with  each  of  the 
signs  in  the  first  row.  The  respective  signal/sign  combinations  and  the  percentage  of 
errors  are  depicted  in  the  ** cells". 


Discussion 

The  velocity  data  support  the  results  of  the  manual  coding  procedure:  Participants  drove 
slower  when  they  had  to  yield  than  when  they  drove  through  the  intersection.  The  low 
overall  error  rate  (3,9%)  indicates  that  subjects  understood  the  task  well.  Interestingly, 
no  difference  in  errors  between  the  age  groups  over  false  yield  and  false  go  responses 
(Table  2)  was  found.  The  low  error  rate  indicates  that  subjects  can  make  efficient  use  of 
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the  presence  of  and  interaction  with  approaching  traffic  to  respond  to  traffic  signals.  For 
a  comparison,  Staplin  and  Fisk  (1991)  obtained  22%  and  23%  false  yield  responses  for 
younger  and  older  adults,  respectively,  whereas  the  corresponding  percentages  were 
9.89%  and  8.85%  in  our  study.  Their  percentage  for  false  go  responses  was  24%  and 
46%  for  younger  and  older  adults,  whereas  we  obtained  0.94%  and  0.31%,  respectively. 
Although  Staplin  and  Fisk’s  subjects  saw  approaching  cars  on  the  still  frames  of  the 
upcoming  intersection  one  might  speculate  that  subjects  might  benefit  from  an  even  more 
realistic,  that  is  dynamic,  environment  as  in  the  current  study. 

Regarding  a  particular  signal  and  sign  combination,  a  closer  look  at  the  error  rates  for 
signal  combinations  with  sign  B  and  D  over  all  drivers  reveals  that  the  error  rates  are 
consistent  with  the  compatibility  hypothesis  described  earlier  in  the  introduction.  As 
expected,  for  signals  implying  a  protected  left  turn  status  (Figure  2)  significantly  more 
errors  were  produced  when  the  sign  implied  a  permitted  left  turn  -  all  combinations  in 
which  both  signal  and  sign  imply  a  protected  left  turn  status  (2D,  5D,  7D)  are  related  to 
lower  error  rates  than  combinations  in  which  the  sign  implies  a  “yield”  or  permitted  left 
turn  status  (2B,  5B,  and  7B).  For  combinations  implying  a  permitted  left  turn  (Figure  3), 
errors  occurred  only  in  condition  ID  (green  globe  combined  with  “protected  left  on  green 
arrow”),  and  in  combinations  IC,  3C  and  6C.  Note  that  no  errors  occurred  with  Sign  A 
and  B  combinations.  Since  Sign  B  is  compatible  with  a  yield  or  stop  response  whereas 
sign  D  implies  a  go  response,  the  result  is  consistent  with  the  compatibility  hypothesis, 
however,  overall  error  rates  are  too  low  to  allow  reasonable  conclusions. 

The  lowest  error  rate  overall  was  found  when  no  sign  accompanied  the  signal  (see  last 
rows  of  Figures  2  and  3).  This  is  also  in  agreement  with  the  results  by  Bonneson  and 
McCoy  (1994)  and  Hummer  et  al.  (1988).  The  data  clearly  indicate  that  in  almost  all 
cases  participants  benefited  from  the  absence  of  a  sign.  An  unexpected  result  was  that 
significantly  more  errors  were  committed  if  sign  C  was  presented  with  a  signal  than  when 
no  sign  was  presented  with  the  protected  left  turn  signals  (Figure  2).  Sign  C  did  not 
relate  to  the  status  of  the  traffic  signal,  however,  almost  as  many  errors  were  produced  for 
combination  5C  as  for  5B  which  produced  the  highest  error  rates  for  a  particular 
signal/sign  combination.  It  appears  that  if  the  meaning  of  the  sign  does  not  directly 
correspond  to  the  meaning  of  the  signal,  its  presence  is  harmful  for  interpreting  the 
signal. 

Conclusions 

The  following  conclusions  can  be  drawn  from  the  above  study: 

•  False  go  responses  are  potentially  much  more  serious  than  false  yield  errors. 
Since  no  false  go  responses  were  made  when  the  sign  was  “left  turn  yield  on 
green  (symbolic  green  globe)”  it  can  be  concluded  that  this  sign  is  beneficial  if 
used  with  a  green  globe  by  itself. 

•  If  a  protected  left  turn  status  is  conveyed  by  the  signal,  the  sign  should  state  so 
explicitly,  too.  Thus,  a  green  arrow  by  itself  can  be  accompanied  by  the  sign 
“protected  left  on  green  arrow”. 

•  In  the  case  that  both  signals  occur  alternately,  the  use  of  variable  message  sign 
technology  seems  obvious:  If  the  signal  is  a  green  arrow  the  sign  should 
convey  a  “protected”  left  turn  status;  if  the  signal  is  a  green  globe  it  could 
convey  a  “yield”  status. 
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•  If  a  protected  left  turn  is  conveyed  by  the  signal  in  combination  with  a 
permitted  left  turn  (green  arrow  and  green  globe),  the  sign  should  not 
emphasize  a  “yield”  message,  i.e.,  the  sign  should  not  be  “left  turn  yield  on 
green  (symbolic  green  globe)”,  A  great  many  false  yield  errors  were  obtained 
if  this  combination  was  used  (Figure  2,  combination  5B).  False  yield 
responses  might  lead  to  an  increased  risk  of  rear-end  collisions;  and 

•  Since  aU  signals  were  best  interpreted  when  no  sign  was  present  it  can  be 
concluded  that  the  absence  of  a  sign  helps  driver  understanding.  This  was 
also  supported  by  the  fact  that  sign  C  which  did  not  relate  to  the  status  of  the 
signal  was  associated  with  many  errors. 
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The  need  for  efficient  simulators  for  training  and  investigating  complex  and  especially 
risky  tasks  is  obvious.  Accordingly  simulators  for  as  diverse  tasks  as  flying  a  plane, 
driving  a  car,  flight  control  and  operating  nuclear  reactors  have  been  developed.  While 
the  benefits  of  simulator  training  are  easy  to  see,  for  instance,  dangerous  situations  can 
be  trained  without  any  risk  for  personnel  of  machinery,  the  question  how  valid  a 
simulation  is,  that  is,  how  similar  is  the  behavior  in  the  simulator  to  the  behavior  in  real- 
life  situations  quite  often  is  left  open  or  only  answered  by  appealing  to  the  obvious 
simularity  in  the  layout  of  instruments  plus  the  physical  characteristics  of  controls  and 
the  realism  of  the  depictured  visual  scene.  Especially,  the  lacking  degree  of  realism  in 
computer  generated  visual  scenes  has  been  criticized:  the  lack  of  non-geometric 
perspective  cues  as  haze  or  blue-shift,  the  unnatural  regularity  of  buildings,  the  crispness 
of  contures,  and,  in  general,  the  lack  of  realistic  clutter  starting  from  pedestrians  on 
seemingly  random  courses  to  debris  and  discolorations  of  the  surfaces.  This  situation  has 
led  to  the  development  of  video-simulators  (2D  and  3D)  at  the  Experimental  Applied 
Psychology  Unit  at  the  University  of  Regensburg,  among  others  (for  an  overview, 
especially  for  research  in  the  US,  see  MacAdam,  1993);  in  this  simulation  methodology 
videos  from  real  traffic  scenes  are  used. 

The  simulator  developed  in  the  Experimental  Applied  Psychology  Unit  at  the  University 
in  Regensburg  consists  of  a  BMW  limousine,  where  all  the  controls  and  displays  are 
linked  to  a  computer,  and  a  video  projector  producing  for  the  driver  a  visual  scene  with  a 
visual  angle  of  about  45°.  It  is  obvious,  that  such  a  kind  of  simulation  is  sensitive  only  to 
the  skills  on  the  basic  level  of  Janssen's  (1979)  hierarchy  of  the  driving  task  (van  der 
Molen  &  Botticher,  1988):  Subjects  can  accelerate  and  decelerate  the  speed  of  the  video, 
and  steering  control  can  be  simulated  by  keeping  a  target  in  the  middle  of  the  lane.  Even 
on  this  level  the  task  is  not  as  interactive  as  in  full  fledged  computer  simulations  because 
deceleration  and  acceleration  influence  not  only  the  simulated  car  in  relation  to  the  stable 
environment  but  also  influences  all  other  moving  objects  in  the  video.  Despite  these 
draw-backs,  the  advantage  of  video  simulation  lies  in  the  realism  of  the  depicted  scenes. 
In  our  experiments  we  were  especially  interested  in  drivers'  speeding  behavior,  for  the 
adjusted  speed  is  the  main  cause  of  traffic  car  accidents. 

In  order  to  determine  the  validity  of  this  simulation  technique  the  correspondences  with 
real-life  driving  have  been  determined  on  three  different  levels  corresponding  to  three 
separable  research  questions: 


236 


(i)  Do  individually  different  driving  styles  induce  a  corresponding  regulation  of  velocity 
in  the  simulation  and  in  the  real-life  driving? 

(ii)  Do  different  driving  tasks  (velocity  maintenance  vs.  self  paced  driving)  lead  to  the 
same  effects  in  those  situations?  and 

(iii)  Do  corresponding  situations  in  the  video  and  in  the  real  world  give  rise  to  the  same 
pattern  of  acceleration  or  deceleration? 

Additionally,  subjective  evaluations  of  both  tasks  have  been  elicited  with  questionnaires 
in  order  to  check  if  they  are  rated  in  such  a  different  way  that  the  experiences  cannot  be 
compared. 

Method 

The  general  experimental  plan  is  shown  in  Table  1, 

Table  1:  _ _  . 


TASK 

SEX 

FIRST  CONDITION 

speed  maintenance 

femaie 

real  life  driving  styie 

si  mu  latord  riving  styie 

maie 

real  life  driving  style 

simulator  driving  style 

free  driving 

femaie 

real  life  driving  style 

simulator  driving  style 

maie 

real  life  driving  styie 

simulator  driving  style 

Additionally,  in  a  questionnaire  data  about  the  subjects"  driving  style  were  obtained.  As 
in  Assmann  (1985)  nearly  all  variance  could  be  attributed  to  three  factors,  namely. 

Factor  1:  following  the  traffic  flow. 

Factor  2:  attitudes  towards  car  driving,  and 

Factor  3:  judging  once  own  driving  ability. 

Using  factor  scores  to  classify  subjects  it  was  possible  to  determine  the  influence  of  the 
driving  style  in  the  two  experimental  situations. 

Table  2  gives  an  overview  over  the  route  used  in  the  experiments;  numbers  indicate  the 
corresponding  situations.  In  order  to  determine  if  accidental  influences  in  the  real-life 
driving  influence  the  choice  of  velocity  any  of  the  following  observations  were 
timecoded:  oncoming  traffic,  following  traffic,  slow  traffic  ahead,  passing  or  being 
passed,  cyclists  on  the  lane,  pedestrians  on  the  lane  or  immediately  beside  the  lane,  and 
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children  close  to  the  road.  The  real-life  driving  was  done  in  a  BMW  730i  with  an 
automatic  transmission.  During  the  field  experiment  all  the  relevant  driving  parameters 
were  entered  into  an  on-board  computer. 


Table  2: 


# 

Speed  limit 

Description 

1 

Segment  of  a  county  road 

60  km/h 

Straight,  flat 

2 

Entering  a  village  with  a  reduced  speed 

zone  (Oberisling) 

60  km/h 

30  km/h 

slightly  curved,  incline 

3 

30  km/h 

slight  decline,  straight 
ahead 

■ 

Leaving  the  reduced  speed  zone 

(Oberisling) 

30  km/h 

60  km/h 

right  turn,  approximately  60° 

6 

60  km/h 

30  km/h 

straight  ahead,  flat 

6 

30  km/h 

60  km/h 

straight  ahead,  flat 

■ 

Entering  the  reduced  speed  zone 
(GraB) 

60  km/h 

30  km/h 

straight  ahead,  priority  of 
the  turning  road 

8 

First  T-intersection  in  GraB 
(without  traffic  signs) 

30  km/h 

9 

Second  T-intersection  in  GraB 
(Sign.'Vield") 

30  km/h 

10 

Free  fane  road 

60  km/h 

50  km/h 

straight  ahead,  slight 
decline 

11 

..Sleeping  policemen"  with  following  cross¬ 
walk  for  pedestrians 

30  km/h 

straight  ahead,  slight 
decline 

12 

..Sleeping  policemen" 

30  km/h 

straight  ahead,  slight 
decline 

13 

Connecting  road 

30  km/h 

straight  ahead,  slight 
decline 

52  subjects  participated  in  the  experiment,  26  female  and  26  male,  in  the  age  range  from 
19  to  26  years  (mean  24,1  years).  Subjects  had  their  driving  licences  between  1.8  and 
1 1.8  years  (mean  6  years).  26  of  the  subjects  were  driving  more  than  10  000  km/year,  26 
less.  Half  of  the  subjects  have  had  experience  with  an  automatic  transmission.  Subjects 
got  a  fee  of  15  DM.  Two  subjects  missed  either  the  simulator  or  the  real-life  driving 
situation,  therefore  their  data  could  not  be  used  in  the  comparison  of  the  corresponding 
behavior. 

The  simulator  consists  of  a  BMW  325i,  where  by  means  of  the  above-described 
projection  technique  driving  can  be  simulated.  Figure  1  shows  what  has  been  simulated 
for  the  speed  regulation  and  what  variables  have  been  skipped. 
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During  simulation  a  virtual  velocity  is  shown  on  the  speedometer  in  the  car.  In  order  to 
influence  the  speed  of  the  video  realistically,  the  pressure  on  the  effectors  in  the  car 
(brakes  and  accelerator)  are  fed  into  a  computer  where  by  means  of  a  formal  car  model 
the  virtual  speed  is  determined  according  to  the  following  formula: 

Vpsd  ='l  (u^*(g-b/u)-u*v^*y)dt 

to 


U 

g 

b 

y 


=  1  -  ( vpsd  /  vmax  ) 
=  go*  1^5 
=  bo  *  4,0 
=  yo  *  0,15  =  0,0  03 


acceleration  depending  on  the  given  velocity 
pressure  on  the  accelerator 
pressure  on  the  brake 
rolling  and  air  resistance 


239 


Due  to  the  fact  that  the  available  videorecorder  (Panasonic  AG  7330)  can  not  increase 
and  decrease  continually  the  following  relation  between  the  virtual  velocity  as  shown  on 
the  speedometer  and  the  projection  velocity  is  as  in  Figure  2. 


Velocity  as  shown 
on  hie  speedometer 
(km/h) 


Speed  of  the  videorecorder 


Figure  2:  Relation  between  pseudospeed  and  recorder  speed 


Results 

In  general,  subjects  regard  driving  in  real-life  situations  as  easier  (median  answer: 
„easy“)  than  in  the  simulator  (median  answer:  ,4’ather  diffucult“).  Furthermore,  the 
simulator  driving  is  regarded  as  needing  more  attention.  Despite  these  differences, 
subjects  when  asked  if  the  simulator  influences  their  driving  behavior,  describe  this 
influence  as  negligable  (modal  answer:  „slightly  if  at  air‘).  For  this  reason  it  can  be 
assumed  that  the  driving  experience  is  comparable  for  both  the  field  and  the  simulator. 

Three  analyses  of  variance  regarding  driving  styles  with  the  factor  scores  of  the 
questionnaire  on  driving  styles  as  the  dependent  variable,  reveal  no  significant 
differences  between  real-life  driving  and  simulator-driving  with  one  exception:  There  is 
an  interaction  between  the  values  of  the  factor  1  („going  with  the  traffic  flow“)  and  the 
driving  situation.  The  Scheffd-Test  reveals  that  subjects  who  „go  with  the  flow“  increase 
the  speed  in  the  simulator  in  contrast  to  the  subjects  with  negative  factor  scores  who 
drive  faster  in  the  real-hfe  situation. 

An  analysis  of  variance  of  the  task  („speed  maintenance'*  vs.  self-paced  driving)  shows 
significant  main  effects  for  the  speed  maxima  and  the  average  speed.  The  task  „speed 
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maintenance**  induces  slower  driving  (53,5  km/h)  with  less  variance  (1,62)  than  self- 
paced  driving  (67,8  km/h  and  2,71  respectively)  but  no  significant  interaction  with  the 
driving  situation  (real-life  vs.  simulator).  Furthermore,  there  is  no  significant  influence  of 
sex  on  the  velocity  regulation. 

Figure  3  shows  the  variability  of  velocity  during  one  test  drive. 


Figure  3:  Curve  of  the  driven  speed  in  real-life  driving  (subject  female,  21  years  old 
with  the  task  „  speed  maintenance  “)  in  relation  to  the  position  on  the  driving 
course. 
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Figure  4  shows  the  corresponding  curve  for  the  simulator  condition. 


Figure  4:  Curve  of  driven  speed  in  the  simulator  in  relation  to  the  driven  course. 

If  one  determines  the  average  maximal  velocity  for  the  13  situations  (see  Table  2)  the 
following  curves  for  real-life  driving  (upper  curve)  and  simulator  driving  (lower  curve) 
result  (see  Figure  5) 


Figure  5:  Mean  values  of  the  maximal  velocities  for  the  13  situations 

For  all  13  situations  (see  Table  2)  and  all  subjects  the  data  for  the  real-life  driving  and  for 
the  simulator  driving  results  of  the  scattergram  in  Figure  6. 
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Virtual  velocity  in  the  simulation 

Figure  6:  Scattergram  of  driven  velocities  per  situation  in  real-life  driving  and  in 
simulator  driving  for  all  subjects  fr  =  .7158;  Regression  (real-life) 
speed  =  12.983  +  .852  *  simulatorspeed 


If  one  reduces  the  noise  and  uses  only  the  mean  velocities  for  the  13  situations  the 
scattergram  of  Figure  7  results. 
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Velocity  in  the  simulation 

Figure  7:  Real-life  driving  velocities  vs,  simulator  driving  velocities  (r=  ,982; 
Regression:  (real-life)  speed  -  6,48  +  1.02  *  simulatorspeed) 

Discussion 

The  data  from  the  questionnaire  where  the  subjects  describe  their  experiences  in  the  real- 
life  driving  situation  and  the  simulator  situation  reveal  that  both  situations  are  regarded 
as  equally  interesting  and  not  influencing  ones^  driving  behavior.  However,  driving  in 
the  simulator  is  regarded  as  more  difficult  and  requiring  more  attention.  The  reason  for 
this  might  be  that  the  simulator  situation  is  novel  and  that  the  velocities  in  the  simulator 
are  not  changed  continuously  but  in  steps.  The  fact  that  driving  in  the  simulator  is 
experienced  as  difficult  and  requiring  special  attention  indicates  that  the  motivation  of 
the  subjects  is  high  even  in  the  simulator  situation.  That  is,  when  driving  in  the  simulator 
subjects  tried  to  exhibit  „normal“  driving  behavior.  This  can  be  seen  not  only  from  the 
subjective  reports  but  also  from  the  fact  that  the  observed  driving  style  does  not  influence 
the  driving  in  the  two  situations  differently  except  for  the  interaction  between  „going 
with  the  flow“  and  average  speed  regulation,  but  also  this  influence  is  very  slight.  The 
conclusion  can  be  drawn  that  in  the  two  situations  there  is  no  differentiating  influence  of 
driving  attituedes  on  the  real-live  vs.  simulator  driving. 

Most  important  for  the  validation  of  the  simulator  are  the  situation  dependent 
correlations  of  situation  specific  velocities  in  the  simulator  and  in  the  real-life  situation. 
Accelerating  and  decelerating  behavior  in  real-life  and  simulator  driving  correspond 
nearly  perfectly.  The  main  influence  on  the  driving  behavior  are  the  situational 
characteristics  independently  from  the  fact  whether  subjects  experience  real-life  driving 
or  simulator  driving. 
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If  one  analyzes  the  individual  correlations  between  the  simulator  data  and  the  real-life 
data  for  the  13  situations,  it  turns  out  that  only  4  subjects  show  correlations  less  than  .5 
but  more  than  75  %  of  the  subjects  show  correlations  above  .85.  The  effect  that  the 
regression  coefficient  does  not  differ  significantly  from  1  shows  that  there  is  nearly  a 
one-to-one  correspondence  between  the  driving  behavior  in  real-life  and  in  the  simulator, 
only  the  additive  constant  of  about  6  km/h  shows  the  influence  of  a  „tunnel  effect  in  the 
simulator. 

Conclusion 

Standard  approaches  for  estimating  the  validity  of  measures  start  from  the  theoretical 
assumption  that  the  „true  validity“  only  depends  on  the  correlation  between  the  „true 
values**  in  the  measure  and  the  „true  values**  in  the  criterion.  These  „true  values**  cannot 
be  measured  directly  and  are  practically  always  confounded  with  the  covariation  of 
systematic  errors.  For  these  reasons  the  selection  of  situations,  measures  etc.  has  to  be 
planned  exactly:  For  instance,  in  the  case  of  the  evaluation  of  our  video  simulator  it  was 
not  important  that  individual  characteristics  of  drivers  („speeding‘*  vs,  „dawdling**  etc.) 
show  in  both  situations  but  that  the  same  situational  factors  elicit  corresponding  kinds  of 
behavior,  that  is,  for  validating  a  simulator  aimed  at  the  improvement  of  traffic 
conditions  the  decisive  carrier  of  information  is  not  the  individual  driver  but  the  specific 
situation,  Generalizability  of  results  therefore  depends  on  the  representativity  of  the 
situations.  Insofar  the  video  simulator  can  be  regarded  as  a  valid  system  for 
determinating  situational  characteristics  of  traffic  safety. 

Since  finishing  this  field  and  simulator  study  the  video  simulator  has  been  further 
improved,  (i)  it  is  now  possible  to  increase  and  decrease  the  projection  velocity 
continually,  (ii)  steering  control  can  be  simulated  additionally  by  shifting  the  video 
picture  to  the  left  or  right  in  accordance  with  a  computerbased  model  of  steering. 
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Abstract 

Modem  cars  will  be  equipped  with  an  increasing  number  of  complex  technical  systems. 
Many  of  these  systems  require  the  driver  to  interact.  Such  an  interaction  is  a  process  of 
controlling  the  subsystems  and  taking  its  response  into  account  reading  a  display.  The 
more  subsystems  will  be  integrated  into  a  car,  the  more  control  devices  and  displays  have 
to  be  placed  in  a  car-cockpit.  In  the  paper  a  variety  of  multifunction  controls  are  de¬ 
scribed,  that  can  be  used  together  with  screen  displays  for  multipurpose  input/output  for 
automotive  communication.  Experiences  are  included.  A  resume  is  given  for  known  as 
well  as  novel  control  concepts 

Introduction 

The  development  of  modem  automobiles  leads  to  more  and  more  complex  technical  sys¬ 
tems.  Vehicles  are  equipped  with  more  sophisticated  systems  onboard.  Traditional  mod¬ 
ules  in  a  car  will  be  electrically  controlled  (windows,  sunroofs,  convertible  roofs  etc.).  All 
these  systems  require  controls  to  operate  them  and  in  many  cases  displays  to  inform  the 
driver  about  their  state  of  operation  (door  locking  state,  radio  frequency,  accessibility  of 
a  cellular  telephone  channel  etc.).  CAN  Bus  and  other  digital  busses  are  used  in  the  elec¬ 
trical  automotive  system.  They  are  capable  of  reducing  the  increasing  number  of  discrete 
controls  as  well  as  the  increasing  cost  and  effort  of  wiring.  Ultimately  the  vehicle  will  be 
a  “drive-by-wire”  system  with  less  mechanical  parts  with  even  steering  wheels,  switches 
to  operate  the  headlights  or  direction  indicators  replaced  by  computers  and  integrated 
controls. 

The  operation  of  traditional  and  more  recent  onboard  systems  using  discrete  mechanical 
controls  may  no  longer  be  feasible  because  spatial  restrictions  and  operational  short¬ 
comings  as  well  as  marginal  designs  with  respect  to  Human  Factors  aspects  are  con¬ 
cerned.  Considerations  to  use  these  controls  not  only  in  the  vehicle  standing  still  but  also 
during  operation  do  even  increase  difficulties  designing  a  control  concept  for  a  modem 
automobile. 

Future  Car-Cockpits 

Flat  screens  are  increasingly  installed  as  multifunction  displays  in  cars.  Presently  they  are 
mainly  used  as  front  ends  for  dedicated  components,  mostly  for  automobile  navigation 
systems.  The  potential  for  flat  screens  as  a  modem  user  interface  for  automobiles  can 
only  be  fully  explored  when  the  screen  will  be  used  as  a  universal  front  end  for  a  larger 
number  of  electronically  operated  automotive  subsystems  (car  radio,  air  conditioning,  on 
board  computer,  telephone  and  functions  to  comfort  the  driver  and  the  passengers).  One 
or  more  flat  screen  displays  in  connection  with  a  carefully  designed  concept  of  control 
devices  will  be  capable  to  replace  dedicated  traditional  controls  by  a  multifunctional, 
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flexible,  and  powerful  Human  Engineered  automotive  cockpit. 


Figure  1:  The  BMW  navigation  system  Figure  2:  The  Blaupunkt  navigation 

system 


New  displays  and  controls  require  space  in  the  automotive  cockpit.  Especially  smaller 
modern  cars  offer  even  less  cockpit  space  than  more  traditional  designs.  Especially 
scarce  is  the  space  for  displays  in  the  so  called  secondary  field  of  vision.  This  field  is 
located  in  the  central  dashboard  area  behind  the  steering  wheel.  It  will  be  called  secon¬ 
dary  field  of  vision  because  the  driver  mainly  monitors  traffic  conditions  outside  the  ve¬ 
hicle  through  the  front  window,  the  true  primary  field  of  vision.  The  secondary  filed  of 
vision  refers  to  the  cockpit  area  that  the  driver  can  monitor  safely  while  the  vehicle  is  in 
motion.  This  area  of  the  cockpit  accommodates  speedometer  and  other  important  dis¬ 
plays  crucial  to  the  function  of  the  automobile.  Tertiary  space  is  available  in  the  center 
console,  mostly  next  to  the  dashboard  controls  and  displays.  The  ever  increasing  number 
and  complexity  of  automotive  systems  require  more  display  space,  especially  in  the  sec¬ 
ondary  field  of  viewing  area.  The  adjacent  controls  must  be  installed  in  those  areas  of  the 
cockpit  as  closely  related  to  the  displays  as  possible.  The  position  called  primary  control 
space  is  in  the  steering  wheel  (or  in  the  future  position  of  the  main  steering  device,  which 
may  be  a  displacement  side  stick  mounted  in  an  arm  rest  to  be  reached  at  ease  by  the 
driver).  The  secondary  display  area  is  practically  unreachable  for  the  driver  because  it  is 
obscured  by  the  steering  wheel.  Even  in  a  future  automobile  this  space  will  not  be  part  of 
the  secondary  control  and  reach  area  because  restraining  harness  systems  will  hinder  the 
operator  to  access  that  area  easily.  The  tertiary  visual  range  is  really  the  secondary  con¬ 
trol  area,  that  is  the  space  in  the  middle  console.  It  is  quite  obvious  that  major  automotive 
systems  will  be  operated  using  controls  in  the  primary  reach  area  and  correspond  to  dis¬ 
plays  in  the  primary  or  secondary  display  space. 

The  described  space  restrictions  make  it  evident  tliat  it  is  not  feasible  to  increase  the 
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number  of  discrete  controls  and  displays  in  a  car.  Integration  of  controls  and  displays  is  a 
vital  interest  for  both  the  manufacturers  and  the  users  of  modem  automobiles.  Yet  the 
integration  has  implications  from  the  technical  side  (aspects  of  production,  installation, 
maintenance,  and  safety  of  operation),  operability  (  function  and  ease  of  operation,  ade¬ 
quate  night  design,  gerontotechnical  aspects),  and  acceptance  by  the  customer  (Human 
Factors  concept,  self  explaining  user  interface,  little  instruction  effort  required). 

Thus  an  integrated  concept  for  an  advanced  control  and  display  system  is  required.  Such 
a  concept  will  have  to  include  multisensory  aspects  for  such  an  Input/Output  system. 
Human  senses  such  as  the  visual  channel,  the  auditory  system,  speech,  and  the  hap¬ 
tic/proprioceptive  capabilities  should  be  included.  The  visual  and  the  auditory  senses 
lend  themselves  for  display  purposes,  the  speech  and  haptic  as  well  as  proprioceptive 
channels  can  both  be  used  controlling  automotive  components  and  systems.  An  inte¬ 
grated  concept  is  far  beyond  the  scope  of  this  paper. 

Multipurpose  Input/Output  for  Automotive  Communication 

In  the  experimental  study  on  which  this  paper  is  based  a  number  of  control  devices  were 
selected  and  tested  to  control  an  automotive  subsystem  similar  to  a  car  navigation  system 
but  not  restricted  to  the  functions  of  such  a  system. 

Speech  input  and  acoustic  output  alone  were  not  considered  because  of  the  complexity  of 
the  task.  Yet,  such  an  approach  is  very  appealing  to  operate  a  cellular  phone  in  a  car  or 
other  dedicated  subsystems  of  comparable  complexity,  hi  this  application  there  is  a  need 
to  input  single  commands.  Phone  operation  requires  only  15  different  commands  for 
comfortable  operation.  For  a  navigation  system,  the  acoustic  system  for  information  in¬ 
put  is  not  flexible  enough.  Moreover,  the  acoustic  systems  may  not  always  work  prop¬ 
erly  if  it  is  very  noisy  during  speech  input.  Acoustic  output  alone  with  no  visual  display 
involved  may  also  unacceptably  restrict  communication. 

Existing  navigation  systems  in  automobiles  use  different  approaches  for  the  Human  Ma¬ 
chine  Interface.  Mercedes  Benz  (Figure  4),  for  example,  uses  function  keys  next  to  the 
screen.  The  BMW  systems  uses  a  twist  button  (Figure  1),  and  the  Blaupunkt  navigation 
systems  (Figure  2)  are  controlled  with  a  so  called  four  way  module.  Some  products  can 
be  mounted  as  add  ons,  not  being  integrated  in  the  dashboard  thus  facilitating  later  in¬ 
stallation  in  older  cars.  All  designs  require  hand  and  arm  movements  towards  the  screen, 
use  cockpit  space  in  the  secondary  control  area  respectively  in  the  tertiary  display  space. 

Control  Devices  for  Use  in  a  Car 

In  the  following  chapter,  control  devices  for  mechanical  information  input  using  hap¬ 
tic/proprioceptive  senses  will  be  described  and  compared.  They  are  always  referred  to  as 
input  devices  in  combination  with  a  flat  screen  display  that  requires  interaction  such  as 
moving  a  cursor  around  or  selecting  from  a  screen  menu. 

There  are  four  main  groups  of  such  devices:  first  there  are  touch-screens  and  touch-pads 
with  or  without  function-keys  next  to  the  screen.  The  next  group  are  rotating  knobs  or 
switches,  also  called  twist  systems.  The  third  group  are  the  so  called  pressure  systems 
generating  inputs  by  pressing  on  a  control  device  surface.  The  fourth  group  are  four-way- 
switches,  a  device  called  EN-Joy  device  and  trackball  systems. 
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A  touch-screen  (Figure  3)  is  basically  an  overlay  mounted  in  front  of  the  screen.  The  user 
touches  the  screen  on  the  desired  location.  There  are  three  different  kinds  of  touch¬ 
screens:  The  capacitance  screen  is  based  on  a  glass  plat  with  a  layer  of  conductive  trans¬ 
parent  material.  Because  of  the  capacity  of  the  human  hand  touching  the  screen,  the  ca¬ 
pacity  between  the  sensors  on  the  layer  will  change.  The  infrared  touch-screen  works 
with  IR-  photoelectric  barriers  witch  are  arranged  in  a  horizontal  and  vertical  position  on 
the  edges  of  the  screen.  A  hand  touching  the  screen  interrupts  at  least  one  x-  and  one  y- 
ray  of  light.  A  controller  locates  the  position.  The  resistive  touch-panel  is  also  based  on 
two  transparent  layers.  In  contrast  to  the  capacitance  technology,  resistance  between  the 
layers  changes  under  pressure. 

A  touch-pad  does  not  work  in  front  of  a  display  like  the  touch-screen.  The  Versa  touch- 
pad,  for  example,  detects  the  touch  position  using  two  semiconductor  layers.  One  layer 
records  the  x-position,  the  other  one  the  y-position.  Touching  the  surface  changes  the 
resistance  between  the  two  layers.  It  is  also  possible  to  register  the  intensity  of  the  touch. 
This  allows  data  input  in  a  “third  dimension”.  There  is  also  a  capacitance  touch-panel 
that  works  like  a  capacitance  touch-screen,  yet  it  is  not  transparent. 

Function  keys,  used  for  example  with  the  Mercedes-Benz  navigation  system,  are  me¬ 
chanical  switches  mounted  next  to  the  screen.  They  are  used  as  multifunction  keys.  Their 
present  functions  will  be  displayed  on  screen.  They  cannot  be  used  for  analog  control. 

Twist  systems  or  rotating  knobs  can  be  distinguished  by  three  different  physical  effects. 
The  paddle-system  (Figure  5)  transfers  it’s  position  to  the  computer  changing  the  resis¬ 
tance  between  two  conductors.  It  has  been  used  as  controller  for  the  first  video  games. 
Paddles  are  analogous  devices.  Their  resistance  tends  to  change  with  time.  For  this  rea¬ 
son  they  cannot  be  used  for  precision  input.  An  additional  problem  is  the  missing  op¬ 
erator  feedback. 

Mechanical  Incremental  Resolvers  (Figure  6)  produce  two  digital  impulses  when  being 
rotated.  The  direction  of  the  spin  can  be  determined  through  the  chronological  order  of 
both  output  signals.  The  advantage  of  the  mechanical  devices  is  that  they  do  not  need 
much  space.  Yet,  they  tend  to  wear  out  when  used  frequently. 

Optical  Incremental  Systems  (Figure  7)  produce  the  same  output  as  the  mechanical  de¬ 
vices.  They  need  a  small  tension  and  a  little  more  space.  The  big  advantage  is,  that  they 
have  nearly  no  attrition.  The  output  is  produced  by  a  light  barrier. 

Pressure  systems  (Figure  8)  are  based  on  two  different  physical  effects:  the  Hall  effect 
and  the  Force  Sensing  Resistor  (FSR).  Combining  four  such  sensors,  a  position  sensing 
system  can  be  realized. 

Four- way- switches  belong  to  the  group  to  multidimensional  sensors.  They  are  applied  to 
adjust  outside  mirrors  in  a  car.  Blaupunkt  uses  them  in  their  navigation  systems  as  well. 
The  technical  principle  is  based  on  a  button,  that  actuates  four  or  more  switches. 
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Figure  3:  The  touch-screen 


Figure  4:  Function  keys 


Figure  5:  paddle  system  Figure  6:  mechanical  incre¬ 
mental  system 


Figure  7:  optical 
incremental  systems 


The  EN-Joy  element,  also  known  as  the  Stick  Switch  from  Matsushita  (Figure  9),  is  a 
device  that  accommodates  different  elements  in  one  system.  It  consists  of  a  mechanical 
incremental  system,  a  confirmation  button  and  a  four-way-switch. 

A  trackball  (Figure  10)  uses  a  sphere  rotating  in  all  directions.  The  sphere  drives  two 
rollers,  one  to  measure  the  motion  of  the  sphere  in  one  direction,  the  x-position,  one  to 
take  the  y-position.  Photoelectric  barriers  measure  the  spin-information  and  turn  them, 
with  the  help  from  a  controller,  into  orientation  signals. 

The  Joy-Three-Dim  device  is  a  small  joystick  accommodating  x-,  y-  inputs  plus  a  z-  co¬ 
ordinate  when  moved  up  and  down  as  well  as  an  angular  information  when  rotated.  It  is 
available  as  a  prototype  and  can  be  moved  with  thumb  and  forefinger  when  installed  in  a 
steering  wheel. 

Based  on  the  construction  size  and  the  function  range,  elements  like  the  Interlink  Point¬ 
ing  device,  the  FID  from  Fujitsu  and  the  GHImouse  enable  efficient  input  strate^es. 
These  devices  also  include  mirror  operation  in  cars.  If  equipped  with  noticeable  switch¬ 
ing  points  these  device  have  a  wide  range  of  automotive  applications. 

Experiences 

The  before  mentioned  control  devices  where  all  installed  in  an  experimental  setup  similar 
to  a  car  cockpit.  The  experimental  task  was  to  operate  a  subsystem  similar  to  a  car  navi¬ 
gation  system  which  was  simulated  on  a  computer  controlled  screen.  A  cursor  had  to  be 
precisely  positioned  on  the  screen  and  menu  items  had  to  be  selected.  The  experiments 
were  not  conducted  and  evaluated  following  a  strict  experimental  design.  The  task  cho¬ 
sen  could  not  be  standardized  at  this  early  point  in  the  development  in  integrated  control 
in  a  vehicle.  The  primary  aim  was  no  direct  ranking  of  the  different  products  but  a 
clearer  understanding  of  their  pros  and  cons  with  respect  to  a  task  similar  to  the  task  cho¬ 
sen.  A  quantitative  ranking  of  the  different  control  setups  did  not  seem  to  make  sense 

Some  input  elements,  like  switches,  have  a  mechanical  feedback  (switching  point).  This 
allows  the  user  to  make  inputs  without  a  direct  optical  control.  So  he  can  devote  his  vis¬ 
ual  attention  to  the  traffic.  Most  of  the  analog  elements  do  not  produce  noticeable  control 
response.  It  was  difficult  to  realize  whether  an  appropriate  input  action  was  taken.  This 
effect  makes  it  particularly  difficult  to  use  such  control  devices  while  the  car  is  moving. 
This  disadvantage  could  be  in  part  compensated  by  allowing  to  use  the  full  range  of  the 
functions  only  when  the  car  was  not  in  motion.  With  the  car  moving  the  available  input 
functions  should  be  reduced.  User  friendly  control  devices,  such  as  touch  screens  could 
lose  their  importance  in  the  future.  The  reason  is  that  screen  displays  could  be  installed 
in  the  secondary  vision  range  replacing  the  speedometer. 

Certainly  those  multidimensional  sensors  will  be  in  demand,  which  can  be  integrated  in 
the  steering  wheel.  A  device  with  many  functions  as  the  joy- three-dim  lead  to  problems 
during  operation.  Most  car  owners  may  not  be  prepared  to  work  with  such  a  complicated 
device.  A  four  dimensional  device  will  require  precise  instruction  before  use  to  avoid 
wrong  input.  Pointing  devices  could  do  more  than  just  mirror  adjustments.  They  are  user 
friendly  and  easy  to  apply. 
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Figure  11:  push  trackball 


The  push  trackball  is  a  patented  trackball  system  (Ohler,  1997).  The  push  trackball  was 
installed  in  the  control  section  a  normal  steering  wheel  (Figure  1 1).  In  the  figure  the  push 
trackball  is  installed  as  the  small  sphere  in  the  second  location  from  the  bottom.  It  works 
much  like  a  normal  trackball  when  operated  by  the  driver’s  thumb.  The  system  works  as 
a  zero-order  or  position  system.  That  means,  that  the  screen  cursor  will  move  when  the 
track  ball  is  rotated  and  stand  still  as  soon  as  the  trackball  is  halted.  The  only  difference 
is  a  switching  function  built  into  the  device.  Pressing  the  ball  down  is  like  pressing  a 
confirmation  button.  The  push  trackball  will  then  send  a  signal  to  the  controller.  Though 
a  bigger  push  trackball  could  be  installed  in  the  middle  console,  the  miniaturized  push 
trackball  in  the  steering  wheel  was  accepted  and  worked  very  well.  The  prototype  showed 
little  haptic  feedback  to  the  driver.  A  later  version  should  be  built  that  has  a  noticeable 
stepping  effect  when  rotated.  Another  version  of  the  trackball  was  tried  out.  This  track¬ 
ball  only  works  in  two  directions,  up-down  and  left-right.  Such  a  system  gives  less  flexi¬ 
bility  but  may  sometimes  prevent  wrong  inputs. 

Conclusions 

Dependent  on  different  applications,  users  found  numerous  the  pros  and  cons  using  dif¬ 
ferent  input  devices  in  an  experimental  setup  similar  to  operating  a  car  navigation  sys¬ 
tem.  S)'stems  installed  in  the  steering  wheel,  that  is  in  the  primary  control  range,  may 
have  advantages  over  devices  installed  elsewhere.  Screen  displays  for  multifunctional  use 
in  a  car  will  probably  be  mounted  in  the  dashboard,  in  the  secondary  visual  range.  Input 
systems  with  many  degrees  of  freedom  can  induce  input  errors  due  to  complicated  mode 
of  operation  and  lack  of  instruction  or  experience  on  the  driver’s  side.  A  two  dimensional 
device  such  as  the  push  trackball  is  easy  to  learn  and  use,  the  push  function  is  acceptable 
and  effective  to  be  used  as  confirmation.  Devices  with  no  proprioceptive  feedback  seem 
to  be  less  acceptable.  Controls  with  no  clear  haptic  characteristics  such  as  a  missing 
feedback  during  operation  are  less  favored.  Combining  more  than  one  input  technology 
can  be  a  great  help  but  was  not  tested.  One  possible  solution  is  a  combination  of  speech- 
input  and  the  push-trackball.  In  this  case  the  trackball  will  be  used  to  interact  on  the  map 
which  is  displayed  on  the  screen,  while  the  speech-input  can  be  used  for  commands. 


Complex  interaction  should  be  conducted  while  the  vehicle  stands  still.  Yet,  there  is  a 
chance  of  safe  interaction  while  the  vehicle  is  in  motion  if  a  control  device  is  installed  in 
the  primary  control  range  being  operable  with  a  finger  while  the  hand  is  securely  rested. 
A  multifunctional  control  strategy  can  be  established  to  control  a  wide  variety  of  on¬ 
board  systems  if  connected  properly  to  a  multifunctional  screen  display  and  an  efficient 
optimized  Human-Machine-Interface. 
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Abstract 

The  Traffic  Research  Centre  developed,  in  co-operation  with  digiTAAL  inc.,  an 
interactive  multi-media  computerprogramme  to  change  speed-limit  offenders’  attitudes 
with  respect  to  speeding.  The  computer  programme  is  meant  to  be  used  during  speed 
controls;  the  offender  may  be  remitted  a  part  of  the  fine  by  completing  the  programme. 
Objective  of  the  programme  is  to  make  speeders  aware  of  the  negative  implications  of 
their  behaviour  and  to  change  their  attitude  negatively  towards  offending  speed  limits. 
To  attain  this  goal,  offenders  are  confronted  with  possible  negative  consequences  of  their 
behaviour  while  their  arguments  for  speeding  are  refuted,  using  small  video-clips, 
demonstrations  of  counter-arguments  and  short  verbal  stories.  The  effects  of  this 
multi-media  programme  were  studied  in  a  laboratory  evaluation,  in  terms  of  knowledge 
and  attitudes,  compared  with  two  information  conditions,  a  general  leaflet  about  traffic, 
and  a  specific  leaflet  about  speeding.  One  week  after  participation  in  the  study  subjects 
were  sent  a  questionnaire  again,  to  measure  whether  changes  in  knowledge  and  attitudes 
were  retained  afterwards.  It  appeared  that  the  general  attitude  towards  speeding  was 
changed  most  in  the  multi-media  programme  condition,  subjects  became  more  negative 
towards  offending  speed  limits  and  various  related  aspects.  The  specific  speeding  leaflet 
appeared  to  influence  the  attitude  towards  driving  fun  positively  and  obeying  traffic  rules 
negatively,  which  are  unwanted  directions.  With  regard  to  knowledge  of  speeding  and  its 
consequences  the  computer  programme  did  not  do  better  than  the  other  conditions. 
However,  the  subjects  considered  the  programme  more  impressive  than  the  leaflet 
conditions  and  indicated  that  they  would  consent  to  participate  when  being  stopped  in 
real  speeding  conditions. 

Introduction 

In  assignment  of  the  Regional  Traffic  Safety  Council  Groningen  (ROG)  and  the 
Ministerial  department  of  Waterworks  and  Road  Maintenance  (RWS)  the  Centre  for 
Environmental  and  Traffic  Psychology  developed  together  with  digiTAAL  inc,  a  multi- 
media  computer  programme  for  driving  speed  enforcement  by  police  and  justice.  Drivers 
who  violate  the  speed  limit  are  halted  and  can  choose  between  the  normal  fine  or 
participating  in  the  programme  and  a  reduced  fine.  Firstly  this  paper  will  summarise  the 
social-psychological  models  that  are  the  basis  for  the  programme.  Secondly,  an 
experiment  will  be  described  that  evaluated  the  programme  in  terms  of  its  ability  to 
change  an  attitude  towards  speeding  in  a  laboratory  set-up,  compared  with  more 
traditional  leaflets.  The  paper  will  be  finished  with  some  conclusions  and 
recommendations. 
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Behavioural  change  with  multi-media  -  a  social-psychological  perspective 

A  hterature  review  was  done  to  find  out  to  what  extent  multi-media  applications  may 
teach  a  different  driving  speed  choice  (De  Rooij,  Werda,  Rooijers  &  Steyvers,  1994).  It 
appeared  that  nothing  was  found,  and  this  did  not  change  since  then.  There  are, 
howerver,  other  ways  to  change  traffic  behaviour,  and  there  are  other  behavioural  goals 
that  were  subject  to  change  by  multi-media  applications.  Concerning  traffic  behaviour  it 
appeared  that  driver  improvement  courses  do  not  (immediately)  change  the  behaviour  of 
problem  drivers,  because  1)  it  is  difficult  to  identify  these  drivers  (Veling,  1986),  and  2) 
the  driver  improvement  programmes  evaluated  had  serious  flaws  (Kaestner,  1981).  These 
flaws  are:  the  programmes  are  far  and  foremost  verbal,  whereas  the  goal  behaviour  is 
sensory-motoric,  only  attitude  was  the  most  important  goal,  thus  underestimating 
external  factors,  accident  involvement  is  not  a  personality  trait,  there  were  no  controls  for 
experimenter  effects  of  placebo  effects,  and  although  a  large  attitude  change  was 
estabhshed,  there  was  no  change  in  behaviour.  In  spite  of  this,  attempting  to  influence 
behaviour  by  influencing  attitude  is  the  only  way  to  try  to  establish  a  "change  from 
within",  since  other  and  proven  more  effective  methods  are  politically,  technically  and/or 
financially  unviable.  Heavy  enforcement  and  fining  campaigns  may  also  bring  about  a 
behavioural  change ,  but  this  is  not  "from  within",  and  will  end  as  soon  as  the  campaigns 
come  to  an  end  (see  e.g,  Rooijers,  De  Waard  &  Soder,  1992). 

The  theoretical  basis  for  a  multi-media  application  to  change  (attitudes  towards) 
speeding  behaviour  comes  from  the  "Theory  of  reasoned  action"  (Fishbein  &  Ajzen, 
1975).  Behaviour  is  thought  to  originate  from  a  behavioural  intention,  which  stems  from 
attitudes  and  subjective  norms.  Attitudes  are  modulated  by  beliefs  about  the 
consequences  of  the  behaviour  in  question,  and  evaluations  about  these  consequences,  in 
this  case  of  speed  in  terms  of  driving  fun,  risks,  travel  time  etc.  The  subjective  norm  is 
the  expected  judgement  of  the  social  environment  and  is  modulated  by  normative 
believes  about  speed  and  by  a  personality  factor  that  may  be  called  tendency  to  comply. 
Some  remarks  are  in  place  here.  This  model  implies  that  drivers  constantly  have 
conscious  control  over  their  behaviour,  which  is  not  so.  Furthermore,  the  model  imphes 
that  behaviour  is  always  the  consequence  of  an  intention,  which  is  not  so.  And  attitudes 
are,  in  this  view,  rational  and  objective,  only  to  be  changed  by  rational  activities,  which 
is  too  narrow-minded. 

To  counteract  these  objections  the  model  had  to  be  extended:  Ronis,  Yates  &  Kirscht 
(1989)  expanded  the  ways  behaviour  is  created.  There  are  two  ways:  reasoned  and 
unreasoned  influences.  Intentions  are  part  of  the  reasoned  influences,  whereas  habits  are 
part  of  the  unreasoned.  Both  these  influences  are  subject  to  modulation  by  internal  and 
external  stimuli  (such  as  infrastructure,  weather,  time  of  day,  reason  for  vehicle  use  etc,), 
and  behaviour  that  originated  from  them  may  be  influenced  by  facilitating  factors  (such 
as  the  availability  of  a  car,  driving  skill  and  knowledge,  etc).  Hence  this  model 
incorporates  the  possibility  for  other  than  rational-conscious  factors  to  influence 
behaviour.  Driving  habits  are  very  strong  behavioural  modulators.  Furthermore  one  may 
argue  that  before  a  trip  is  taken  various  social  factors  may  influence  behavioural 
intentions,  but  once  in  the  "wheeled  cage"  the  possibilities  for  influencing  drivers  by 
social  factors  diminish  strongly  (ways  to  communicate  with  drivers  by  other  drivers  are 
minimal),  and  hence  within  a  certain  task  and  environmental  context  habits  take  over. 
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Ronis  et  al.  however  maintain  that  habit  change  only  will  take  place  when  repeatedly 
conscious  information  processing  takes  place.  The  driver,  however,  will  resist  reasons  for 
other  behaviour.  Petty  &  Cacioppo  (1986)  incorporate  this  resistance  to  change  in  their 
"Elaboration  likelihood  model".  This  model  has  two  routes  by  which  behaviour  may 
change.  When  the  driver  is  willing  and  able  to  do  conscious  and  rational  information 
processing,  persuasion  of  the  driver  towards  the  wanted  behaviour  is  possible  by  a  direct 
or  central  route  of  rational  information.  When  the  driver  is  not  willing  and/or  able  to 
consciously  process  rational  information  a  kind  of  detour,  a  peripheral  route  has  to  be 
taken.  In  stead  of  a  rational  an  emotional  and  associative  approach  has  to  be  taken. 

In  case  of  speeding  behaviour  all  attempts  to  change  behaviour  can  be  expected  to  meet 
certain  resistance.  Speeders,  halted  in  an  enforcement  campaign,  are  expecting 
beforehand  attempts  of  influence  and  arm  themselves  mentally  with  arguments  in  favour 
of  their  (unwanted)  behaviour.  Influencing  them  by  taking  the  peripheral  route  may  be 
more  successful,  using  associations,  and  emotionally  loaded  arguments.  Using  a 
computerised  multi-media  programme  it  may  be  an  advantage  that  the  speeder  may  be 
actively  involved  in  handling  the  application  (interactive  approach),  that  more  or  less 
individual  courses  through  the  programme  are  possible,  an  that  the  emotional  impact 
may  be  augmented  by  using  various  media  (sound  bites,  short  films,  pictures  of  the 
actual  situation  of  the  offence  etc.). 

The  basis  of  the  actual  programme  is  formed  by  eleven  reasons  speed  offenders  may 
think  of  for  their  behaviour:  I  was  in  a  hurry,  speeding  is  fun,  I  was  behind  a  snail,  this 
speed  is  best  for  my  car,  I  was  not  aware  of  my  speeding,  I  did  not  know  you  the  actual 
limit,  the  speedometer  was  broken,  I  don’t  accept  the  local  hmit,  I  decide  my  driving 
speed  myself,  all  drivers  driver  more  than  the  limit  here,  and  there  is  never  a  speed 
control.  Each  of  these  is  counteracted  by  both  rational  and  emotional  arguments,  by  short 
and  hefty  pictures,  films  or  sounds.  Furthermore  the  offender  is  confronted  with  the 
negative  outcome  of  speeding  behaviour  by  him/herself,  by  engaging  him/her  in  various 
tasks,  such  as  a  reaction  time  task,  brake  distance  estimation  task  and  the  like. 

A  laboratory  evaluation 

To  find  out  whether  the  programme  is  better  in  changing  attitude  than  traditional  leaflets 
an  evaluation  was  performed.  A  situation  was  invented,  a  50-km/h  road  running  through 
a  suburban  village.  The  situational  specific  frames  in  the  programme  were  equipped  with 
pictures  and  accident  information  of  the  location.  There  were  three  conditions  for  the 
between -subject  study:  the  programme,  a  specific  speed-related  leaflet,  and  a  general 
traffic- safety  leaflet.  In  each  condition  20  subjects  were  scheduled:  experienced  drivers 
that  once  or  even  more  often  were  confronted  with  a  fine  for  speeding,  57  subjects 
actually  participated  in  the  study  (33  male,  24  female):  18  in  the  programme  condition, 

1 9  in  the  specific  leaflet  condition,  and  20  in  the  general  leaflet  condition.  After  an  oral 
introduction  subjects  had  to  finish  a  questionnaire  for  assessing  their  attitude.  Then  they 
were  confronted  with  their  condition  (doing  the  programme  or  reading  the  leaflets).  Then 
they  had  to  finish  a  questionnaire  again.  One  week  after  participation  in  the  lab  a 
questionnaire  was  mailed  to  them,  for  assessing  their  attitude  again. 

The  attitude  questionnaire  consisted  of  three  bi-polar  five-point  scales  (from  -2  to  +2) 
anchoring  with  good  -  bad,  fun  -  no  fun,  attractive  -  unattractive.  The  composite 
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(summated)  score  is  interpreted  as  the  attitude  towards  speeding.  Table  1  shows  these 
summated  scores,  separately  for  each  condition  and  the  pre-  and  post- treatment 
assessment.  As  can  be  seen  the  attitude  in  the  programme  group  towards  speeding  is 
significantly  decreased  (speeding  became  worse,  less  fun  and  less  attractive),  whereas  for 
the  two  other  groups  there  was  no  (significant)  change.  Furthermore,  the  difference 
between  the  programme  group  and  the  specific  leaflet  group  was  significant  in  the  post¬ 
treatment  assessment:  the  programme  group  became  more  negative  about  speeding, 
whereas  the  specific  leaflet  group  did  not. 

Table  1:  Summative  attitude  scores  for  each  of  three  treatment  groups,  separately  for  pre- 
treatment  and  post- treatment  assessment.  Scores  range  from  -2  to  +2.  *  =  significant 
difference  between  pre-  and  post-treatment  assessment. 


Treatment  group  pre-treatment 

assessment 

I  Multimedia  Programme  0.0 

n  Specific  Leaflet  0.4 

in  General  Leaflet  -0.6 


post-treatment 

assessment 

-1.1  (*) 

0.3 

-0.2 


There  were  also  questions  about  motives.  The  following  aspects  were  assessed:  agreeable 
driving  speed  for  the  vehicle,  driving  fun,  having  self-control  of  the  speed,  the  vehicle 
remains  "speedy",  endangering  other  drivers,  traffic  rule  compliance,  lacking  time  to 
react  on  other  drivers’  behaviour,  probability  of  a  speeding  fine,  incorrect  estimation  by 
other  drivers  of  the  subject’s  speed,  gaining  time,  being  quickly  at  the  destination.  Each 
of  these  aspects  was  expressed  as  a  short  statement.  Subjects  had  to  indicate  their 
measure  of  agreement  with  these  statements  (from  -2  to  +2)  and  their  measure  of 
considering  this  important  (also  from  -2  to  +2).  The  product  of  these  scores  provides  a 
score  for  behavioural  motive.  Table  2  shows  these  scores,  for  each  of  the  treatment 
groups,  separately  for  pre-  and  post- treatment  assessments. 

Table  2:  Mean  scores  for  the  behavioural  motives  of  the  three  treatment  groups, 
separately  for  pre-  and  post-treatment  assessment.  Scores  range  is  from  -4  to  -i-4.  A  high 
score  means  an  advantage  for  speeding. 

Subjects  in  the  various  treatment  groups  appeared  to  differ  significantly  in  their  opinion 
about  various  aspects.  Since  this  was  an  indicative  study  the  significance  level  was 
placed  at  0.10  in  stead  of  the  usual  0.05.  The  specific-treatment  group  in  the  pre¬ 
treatment  assessment  considered  driving  fun  more  negative  than  the  programme  group 
(t=1.70,  df=35,  p  <  0.10).  The  general-leaflet  group  was  more  negative  about  probability 
of  a  speeding  fine  (t  =  2.39,  df  =  37,  p  <  0.05)  and  about  the  vehicle  remaining  more 
"speedy"  (t  =  1.70,  df  =  37,  p  <  0.10).  Subjects  of  the  programme  group  considered  the 
aspect  being  quickly  at  the  destination  more  positive  than  subjects  from  the  general 
leaflet  group  (t  =  2.05,  df  =  36,  p  <  0.05).  In  the  post-treatment  assessment  subjects  of 
the  specific  leaflet  group  considered  the  vehicle  remaining  more  "speedy"  a  more 
positive  aspect  of  speeding  than  the  subjects  of  the  general  leaflet  group  (t  =  1.80,  df  = 
36,  p<  0.10). 
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Pre-treatment  assessment 
post-treatment  assessment 


Treatment  group  I 

n 

m 

I 

n 

m 

Motive 

Best  speed 

for  the  car  0.7 

0.7 

1.1 

0.8 

0.8 

1.2 

Having  driving 
fun 

-0.6 

-1.6 

-0.5 

-0.4 

-0.3 

0.4 

Self  determination 
of  speed  1.1 

0.5 

0.2 

0.6 

0.3 

0,5 

Car  remains  more 
’speedy’  1.2 

1.4 

0.3 

1.4 

1.9 

0.9 

Endangering  other 
drivers  -0.7 

-0.5 

-1.1 

-0.4 

-0.7 

-0.8 

Compliance  of 
traffic  rules  -1,4 

-1.9 

-0.9 

-0.4 

-0,8 

-0.8 

Lacking  time  to  react 
to  others’  behaviour-0.2 

-0.2 

-0.6 

-1,3 

-1.9 

-0.9 

Probability  of  speeding 
fine 

-2.6 

-1.9 

-3.3 

-1.9 

-1.8 

-2.1 

Incorrect  speed  estimation 
by  others  -0.8 

0.2 

-0.9 

-1.2 

-1.1 

-0.7 

Gaining 

time 

0.8 

0.3 

0.2 

0.1 

0.5 

0.6 

Being  at  destination 
more  quickly  1.6 

0.6 

0.5 

1.0 

0.8 

0.7 

Then  the  change  between  pre-  and  post-treatment  assessments  were  compared.  Subjects 
in  the  programme  group  became  less  positive  about  the  aspect  of  gaining  time  (t  =  1.80, 
df  =  17,  p  <  0.10).  Subjects  of  the  specific  leaflet  group  considered  the  aspect  diving  fun 
in  speeding  more  positive  (t  =  2.09,  df  =  18,  p  <  0,10)  and  traffic  rule  compliance  more 
negative  (t  =  2.52,  df  =  18,  p  <  0.05)!.  They  became  more  negative  about  the  time  to 
react  to  other  drivers’  behaviour  (t  =  2.51,  df  =  18,  p  <  0.05),  and  about  the  incorrect 
estimation  by  other  drivers  of  the  subject’s  speed  (t  =  2.05,  df  =  18,  p  <  0.10).  Subjects  of 
the  general  leaflet  group  considered  driving  fun  more  positive  (t  =  2.01,  df  =  19,  p  < 
0.10),  as  well  as  probability  of  a  speeding  fine  (t  =  3.01,  df  =  19,  p  <  0,01)  and  gaining 
time  by  speeding  (t  =  1,76,  df  =  19,  p<  0.10). 

There  were  questions  concerning  the  specific  situation  that  was  well  known  to  all 
subjects.  They  considered  the  speed  limit  of  50  km/h  quite  sensible,  and  they  thought 
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that  traffic  safety  would  increase  when  all  drivers  would  comply  to  that  limit.  The 
statement  "I  comply  to  a  limit  earlier  when  accompanied  in  the  vehicle  than  when  alone 
gave  varied  levels  of  agreement,  hi  the  post-treatment  assessment  subjects  of  the  specific 
leaflet  group  more  often  were  totally  opposed  the  statement  (chi- square=  13,85,  df=8,  p  < 
0.10),  compared  to  the  other  groups.  Subjects  from  the  general  leaflet  group  and  the 
programme  group  considered  police  speeding  control  more  sensible  than  subjects  from 
the  specific  leaflet  group.  This  was  found  both  in  the  pre-  and  in  the  post- treatment 
assessment  (respectively  chi-square  15.11,  df  6,  p  <  0.05  and  chi-square  11.44,  df  =  6,  p 
<  0.10),  hi  the  post-treatment  assessment  the  difference  became  smaller.  All  subjects 
estimated  the  probability  of  a  speeding  fine  on  the  specific  location  rather  small. 
Unnoticed  one  tends  to  drive  faster  than  allowed.  A  speeding  fine  was  considered  very 
unpleasant.  Subjects  reacted  variously  when  asked  wether  on  would  driver  faster  if  speed 
controls  were  certainly  not  to  be  held. 

In  pre-  and  post-treatment  assessments  also  knowledge  was  tested  by  asking  questions 
about  the  following  aspects:  proportion  of  drivers  that  complied  to  the  speed  limit  at  the 
specific  (fake)  location,  liability  in  case  of  an  accident  with  a  bicyclist,  insurance  policies 
in  this  case  (loss  of  no-claim  reductions),  braking  distance  at  50  km/h,  braking  distance 
at  80  km/h,  physical  impact  of  a  collision  with  50  km/h  in  terms  of  a  free-fall  jump, 
meaning  of  traffic  sign  "end  of  built-up  area"  for  speed  limit,  estimation  of  time  gained 
by  a  speed  of  20  km  above  the  limit  at  a  stretch  of  10  km,  magnitude  of  the  fine  in  case 
of  a  speed  of  15  km  above  the  limit,  speed  over  the  limit  that  may  cause  the  drivers 
licence  to  be  withdrawn.  In  the  pre-treatment  assessment  there  were  no  differences  in 
knowledge  between  the  three  treatment  groups.  In  the  post-treatment  assessment  there 
were  two  differences.  The  subjects  of  the  specific  leaflet  group  and  (less  pronounced)  of 
the  programme  group  responded  more  often  that  the  drivers  licence  might  be  withdrawn 
at  speed  limit  violations  between  31  and  60  km/h  (chi-square  13.24,  df  =  6,  p  <  0.05). 
Forty-five  percent  of  the  respondents  in  the  general  leaflet  group  thought  (correctly)  that 
a  non-guilty  collision  with  a  bicycle  would  cause  the  loss  of  the  no-claim  insurance 
reduction,  whereas  in  the  programme  group  and  the  specific-leaflet  group  this  was 
seventeen  and  thirty  three  percent  respectively  (chi-square  8.23,  df  =  4,  p  <  0.10).  All 
respondents  underestimated  in  both  pre-  and  post- treatment  assessments  the  proportion 
of  drivers  complying  to  the  speed  limit.  Most  drivers  were  aware  of  the  liability  in  case 
of  a  non-guilty  collision  with  a  bicycle,  but  that  this  would  mean  the  loss  of  the  no-claim 
reduction  of  the  insurance  was  not  common  knowledge.  The  meaning  of  the  sign  "end  of 
built-up  area"  in  terms  of  a  speed  limit  of  80  km/h  unless  otherwise  stated  was  common 
knowledge.  And  subjects  estimated  the  time  gained  by  speeding,  and  the  fine  in  case  of  a 
speed  limit  violation  of  15  km/h  quite  accurately  Fur&ermore  there  was  a  systematic 
underestimation  of  braking  distances  for  speeds  of  50  and  80  km/h.  Finally  subjects 
underestimated  the  speed  limit  violation  at  which  police  officers  may  withdraw  the 
drivers  licence. 

To  gain  information  about  the  medium,  especially  the  multi-media  programme, 
immediately  after  the  treatment  a  questionnaire  was  filled  in.  Subjects  had  to  rate  on  a 
five-point  scale  (good-bad)  the  following  aspects  of  the  topics  treated:  educational-not 
educational,  understandable-not  understandable,  clear-unclear,  making  sense-not  making 
sense,  good-bad,  boring-exiting,  realistic-unreahstic,  fun-no  fun.  And  they  had  to  rate  the 
way  the  topics  were  presented  with:  fun-no  fun,  understandable-not  understandable, 
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making  sense  -  not  making  sense,  good-bad,  real- artificial.  Subjects  were  more  positive 
about  the  programme  than  about  the  leaflets  (chi-square  12.30,  df  =  6,  p  <  0.10). 
Furthermore  subjects  of  the  programme  group  tended  towards  a  more  positive  judgement 
about  the  contents  of  the  programme.  They  considered  it  more  understandable,  exiting, 
and  realistic.  On  other  aspects  the  programme  scores  as  good  as  the  leaflets,  with  the  one 
exception  that  the  general  leaflet  was  more  "making  sense"  than  the  other  treatments. 
This  positive  rating  of  the  programme  was  also  found  for  the  aspects  of  treatment  of  the 
topics.  The  specific  leaflet  however,  was  found  more  understandable  than  the  other  two 
treatments.  In  general,  the  multi-media  programme  was  appealing  to  the  subjects.  About 
the  length  subjects  were  diverted.  Two  considered  it  too  long,  nine  neither  too  long  nor 
too  short,  and  seven  too  short,  one  remarking  "a  punishment  in  terms  of  time  is  horrible 
for  speeders".  Executing  the  computer  was  no  problem.  Given  the  choice  between  the 
programme  and  the  full  fine  most  (16  of  18)  subjects  would  take  the  programme.  Finally 
details  of  the  programme  were  given  ratings  and  subjects  gave  open-end  remarks  about 
anything  they  would  like  to  comment  on. 

Discussion  and  conclusions 

The  results  of  this  evaluation  study  may  be  summarised  as  follows.  The  multi-media 
computer  programme  for  educational  enforcement  of  speed  limit  offenders  did  bring 
about  a  change  in  attitude  towards  driving  too  fast.  It  became  significantly  more 
negative,  whereas  subjects  treated  with  a  more  traditional  medium  -  specific  or  general 
leaflets  -  did  not  show  such  an  attitude  change.  Subject  in  the  general  leaflet  group  even 
showed  a  slightly  more  positive  attitude  towards  speeding,  and  subjects  in  the  specific 
leaflet  condition  remained  more  or  less  positive.  Furthermore  opinions  about  various 
aspects  of  speeding  were  assessed.  The  programme  caused  subjects  to  be  less  positive 
about  the  gain  of  time  by  speeding.  Subjects  are  now  aware  that  speeding  does  not 
automatically  result  in  gain  of  time.  The  specific  leaflet  brought  about  that  subjects 
became  less  negative  about  driving  fun  by  speeding,  and  less  positive  about  complying  to 
traffic  rules.  This  is  in  contrast  with  what  one  would  want  to  achieve  in  such  treatments. 

The  programme  group  did  not  gain  more  knowledge  about  the  consequences  of  speeding, 
compared  to  the  other  two  treatment  groups.  It  appears  that  subjects  already  possess  quite 
some  knowledge  in  this  respect.  However  the  way  the  various  topics  are  treated  was 
judged  more  positive  for  the  programme  than  for  the  other  treatments.  This  was  the  case 
in  this  laboratory  experiment.  In  a  real-life  setting  the  advantage  of  the  programme  over 
leaflets  may  become  even  larger,  hi  this  experiment  subjects  were  actually  reading  the 
leaflets,  whereas  in  real  life  people  take  the  leaflet,  put  it  away  in  order  to  drive  on,  and 
forget  its  existence. 

Therefore,  on  may  conclude  that  the  programme  may  contribute  to  a  change  in  attitude 
towards  driving  speed  violations  in  general,  and  about  the  "myth"  of  time  gained  by 
speeding  in  particular.  The  question  whether  this  treatment  not  only  will  cause  a  change 
in  attitude  but  also  in  driving  behaviour  is  the  next  step  in  the  evaluation  of  the 
programme.  Field  trials  are  planned  on  both  50  and  80  km/h  roads. 
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Abstract 

A  Head-Up  Display  (HUD)  is  an  optical  control  which  is  mirrored  in  the  windscreen  of  a 
car.  The  apparent  distance  of  this  kind  of  display  can  be  varied  with  the  help  of  the 
optical  system  used.  HUDs  can  be  used  to  offer  information  to  the  driver  without  forcing 
him  to  accommodate  to  the  instmment  panel.  This  reduces  the  time  the  driver  needs  to 
perceive  the  presented  information.  Experiments  with  Head-Up  speedometers  show  that 
subjects  prefer  using  the  HUD.  After  a  short  period  of  habituation  the  still  available 
normal  speedometer  is  used  very  seldom. 

HUDs  can  also  be  used  to  display  additional  information:  For  example  the  distance  a 
vehicle  moves  during  the  reaction  period  of  the  driver. 

Actual  experiments  examine  the  use  of  HUDs  for  new  navigation  assistance  systems.  In 
this  case  the  information,  normally  presented  on  a  small  screen  in  the  car  or  as  acoustical 
information,  is  presented  directly  in  the  environment  and  is  perceived  similar  to  road 
signs  or  traffic-line  markings. 

Because  of  reproducible  traffic  situations,  short  implementation  time  for  new  display 
types  (they  must  simply  be  added  to  the  simulation  scene)  and  the  possibility  of 
examination  of  even  dangerous  situations  with  normal  subjects,  statements  based  on 
statistics  can  be  made  faster  than  in  not  simulated  experiments. 

In  this  presentation  the  actual  research  and  results  will  be  presented  with  an  additional 
short  video  demonstration. 

Introduction 

Looking  at  the  control  panel  to  read  the  actual  speed  or  other  information  about  the  car  is 
something  the  driver  of  a  car  does  very  often.  Normally  the  driver  has  to  look  away  from 
the  outside  traffic  situation  in  order  to  perceive  the  indicated  information.  To  reduce  the 
time  a  driver  is  turning  away  his  attention  from  the  outside  some  car  manufacturers  made 
studies  whith  HUDs  -  which  were  used  in  air  planes  for  a  long  time  -  in  their  vehicles. 
Although  the  first  prototypes  for  car  HUD  systems  were  realized  more  than  ten  years  ago, 
only  a  few  companies  integrated  simple  HUDs  in  their  cars  on  the  market  (e.g.  Buick 
ParkA venue,  Nissan  Bluebird  and  Pontiac  Bonneville).  Many  of  the  other  companies 
regard  the  HUD  technique  as  a  topic  of  basic  research  for  future  developments.  They 
don’t  believe  that  the  HUD  offers  a  security  enhancement  even  if  there  are  a  lot  of  studies 
which  show  the  advantages  of  the  HUD  technique.  Some  of  these  mentioned  by 
Schneider  et.  al.  (1992)  are: 
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•  The  driver  perceives  the  indicated  information  in  a  smaller  viewing  angle  to 
the  traffic  situation. 

•  There  is  less  or  in  some  cases  even  nearly  none  accommodation  between  near 
and  far  distances  needed  to  read  the  indicated  information. 

•  Adaptation  to  different  lightning  conditions  on  the  board  instruments  and  the 
outside  of  the  car  are  no  longer  necessary. 

All  these  items  lead  to  shorter  periods  of  time  in  which  the  driver  is  turning  his  attention 
from  the  traffic  situation.  For  example  Bartolomai  (1990)  found  that  the  time  to  read  the 
speedometer  in  different  conditions  can  be  reduced  from  a  range  of  100  to  2000  ms  to  a 
range  between  100  and  800  ms  by  using  a  HUD  speedometer.  As  an  interesting  detail 
about  HUDs  Bubb  (1992)  mentioned  the  fact  that  air  planes  with  this  technique  have  to 
pay  less  insurance  fee,  because  of  the  improved  security  level. 

How  an  HUD  works 


Figure  1:  The  basic  implementation  of  a  HUD 

The  basic  implementation  of  a  HUD  as  shown  in  figure  1  is  always  the  same:  An 
ordinary  display  consisting  of  any  source  of  light  (e.g.  an  arrangement  of  LEDs  or  an 
TFT  display)  is  mirrored  in  the  windscreen  of  a  car  by  an  optical  system.  The  optical 
system  must  be  adjustable  to  the  viewpoint  of  the  driver,  so  that  the  user  perceives  the 
HUD  at  the  right  position  behind  the  windscreen.  The  distance  of  the  virtual  image  of  the 
HUD  depends  on  the  optical  system  used.  For  simple  HUDs  often  a  distance  between  2.5 
and  3.5  meters  is  used.  This  distance  is  easy  to  realize  and  works  for  all  standard 
information  such  as  speed,  rounds  per  minute,  remaining  fuel  and  motor  temperature 
warnings.  An  disadvantage  of  these  simple  HUD  systems  is  that  the  HUD  seems  to  move 
in  relation  to  the  outside  traffic  situation  whenever  the  driver  moves  his  head.  Therefor 
this  simple  system  has  to  be  replaced  by  another  with  a  variable  or  a  larger  apparent 
distance  of  the  HUD  to  indicate  information  that  should  be  combined  with  the  outside 
traffic  situation  by  the  driver.  A  HUD  that  offers  such  information  is  called  a  contact 
analogue  HUD.  Some  applications  of  such  HUD  systems  will  be  described  later. 

Our  own  experiences  using  HUDs 

In  our  institute  there  where  a  lot  of  studies  related  to  HUDs.  Most  of  them  where  done 
with  HUDs  in  real  vehicles.  These  studies  concerned  with  HUDs  which  had  a  fixed 
apparent  distance  to  the  driver.  Most  of  the  studies  like  those  of  Assmann  (1985)  and 
StUrzer  (1983)  were  about  a  new  HUD  concept  that  allows  to  indicate  the  distance  a  car 
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moves  during  the  reaction  period  of  the  driver.  This  HUD  with  an  apparent  distance  of 
eight  meters  consisted  of  a  rectangle  that  could  be  varied  in  si2e  and  height  to  simulate  a 
green  bar  lying  on  the  road  in  the  desired  distance.  The  apparent  distance  of  eight  meters 
was  enough  to  achieve  a  sufficient  realistic  impression.  As  a  base  of  these  studies 
Bierbrauer  already  did  studies  in  an  other  simulator  with  this  concept  in  1980. 

Other  studies  at  our  institute  dealt  with  the  habituation  of  subjects  to  a  HUD  speedometer 
similar  to  the  one  shown  in  figure  2,  In  these  studies  our  line  of  sight  detection  system 
JANUS  was  used  to  get  information  about  how  often  and  how  long  a  subject  looks  at  the 
HUD  and  the  still  available  ordinary  speedometer,  Gengenbach  (1997)  showed  that  all 
subjects  in  this  study  preferred  using  the  HUD  speedometer.  He  also  confirmed  the 
shorter  time  that  is  needed  to  read  a  HUD  with  his  evaluation. 


Figure  2:  Current  implementation  of  a  HUD  speedometer  in  our  simulator. 

The  simulator  as  a  tool  to  evaluate  new  HUD  concepts 

A  lot  of  our  studies  about  HUD  systems  tried  to  show  the  advantages  of  contact  analogue 
HUDs.  To  obtain  the  best  possible  quality  for  this  kind  of  display  there  were  many 
experiments  with  adjustable  optical  systems.  Gengenbach  (1993)  even  made  studies  with 
a  holographic  HUD.  A  result  of  all  these  studies  was,  that  there  is  a  lot  of  engineering 
necessary  to  realize  a  HUD  system  that  is  able  to  indicate  an  contact  analogue  HUD  for  a 
special  purpose.  In  addition  an  optical  system  that  worked  good  with  one  HUD  concept 
often  couldn’t  be  used  with  others.  The  free  programmable  TFT  display  used  by  many 
vehicle  manufacturers,  which  is  very  flexible  to  examine  new  standard  HUD  concepts  are 
also  not  suitable  for  these  kind  of  studies.  One  way  to  generate  a  tool  that  makes  it 
possible  to  examine  the  mammachine-interface  of  new  contact  analogue  HUDs  without 
having  to  develop  a  complicated  optical  system  that  is  able  to  indicate  this  HUDs  in  the 
desired  way  was  to  use  virtual  HUDs  in  a  simulator. 

By  using  a  virtual  HUD  it  is  also  much  easier  to  examine  concepts  that  normally  would 
cause  a  lot  of  expense.  For  example  a  HUD  prototype  for  a  navigation  assistance  system 
that  normally  should  be  linked  to  an  on  board  GPS  (Global  Positioning  System)  or  for  a 
driver  assistance  system  that  indicates  warnings  if  the  vehicle  gets  to  near  to  the  roadside 
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or  if  it  gets  to  close  to  the  car  ahead  can  be  implemented  by  using  the  already  available 
information  about  the  positions  of  all  objects  in  the  simulated  scene  (including  other  cars) 
and  an  additional  database  that  includes  the  rights  of  way  at  road  junctions  or  road 
crossings,  the  speed  restrictions,  information  about  traffic  signs  and  the  suggested  courses 
for  a  navigation  assistance  system.  It  is  also  much  easier  to  realize  completely  new  ideas 
like  an  assistance  system  for  parking  between  two  other  cars  where  a  HUD  could  be  used 
to  visualize  the  distances  to  other  cars,  traffic-line  markings  and  other  obstacles  in  all 
directions  of  a  car  without  having  to  integrate  a  lot  of  measurement  instruments  in  a 
prototype  vehicle. 

The  simulator  offers  a  wide  palette  of  possibilities  to  implement  HUD  systems.  The 
virtual  HUDs  can  be  moved  in  relation  to  the  car  by  using  DCSs  (Dynamic  Coordinate 
Systems),  the  indicated  HUD  can  be  modified  in  size,  color,  transparency  and  brightness 
by  using  morph  nodes  and  last  not  least  it  is  possible  to  implement  HUDs  with  different 
states  such  as  the  digits  of  a  speedometer  by  using  switch  nodes.  For  more  information 
about  the  graphical  abilities  of  a  simulation  system  it’s  recommendable  to  have  a  look  at 
a  handbook  for  a  simulation  software  like  the  IRIS  Performer  library  by  SGI. 

Examples  of  contact  analogue  HUDs  implemented  in  a  simulator 
The  first  contact  analogue  HUD  that  was  integrated  in  our  new  simulator  was  the  already 
mentioned  security  distance  indicator.  To  be  able  to  compare  the  virtual  HUD  with  the 
one  we  examined  in  a  real  car  we  did  two  different  implementations.  The  first  was  a 
green  rectangle  with  an  fixed  distance  of  eight  meters  that  could  be  varied  in  height  and 
size  to  get  nearly  the  same  impression  as  with  the  display  that  was  implemented  in  the 
real  car.  The  second  implementation  was  a  simple  rectangle  (shown  in  figure  3)  which 
wasn’t  modified  in  height  and  size,  but  was  lying  on  the  road  and  moved  to  the  point  of 
the  road  where  the  car  would  be  after  the  reaction  period  of  the  driver. 


Figure  3:Contact  analogue  security  distance  HUD 


The  main  advantages  of  the  second  implementation  were  that  head  movements  of  the 
driver  didn’t  result  in  apparent  movements  of  the  indicated  HUD  in  relation  to  the  road 
surface  and  that  the  HUD  didn’t  had  to  be  adjusted  to  the  viewpoint  of  the  driver. 
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Another  HUD  project  in  our  simulator  is  about  navigation  assistance  systems.  These 
systems  are  expected  to  show  the  driver  where  he  should  turn  left  or  right  in  order  to  get 
to  a  spotted  point  of  the  scenery.  A  simple  and  quite  good  possibility  for  such  a  system 
would  be  another  car  which  leads  you  to  the  selected  location.  As  it  is  not  possible  to 
indicate  a  leading  vehicle  in  a  HUD,  because  this  display  would  hide  to  much  of  the 
surrounding  and  would  take  to  much  of  the  attention  of  the  driver,  an  more  simple  HUD 
had  to  be  implemented  which  offered  nearly  the  same  comfort.  A  first  implementation  as 
shown  in  figure  4  was  a  simple  HUD  which  indicated  the  navigation  systems  advises  in 
an  symbolic  way. 


Figure  4:  Simple  HUD  for  navigation  control  information 
This  HUD  system  was  sufficient  in  most  traffic  situations.  Only  if  there  is  a  short 
succession  of  road  junctions  the  indicated  information  might  not  be  clear.  To  improve 
this  HUD  a  contact  analogue  HUD  was  realized  which  consisted  of  a  set  of  arrows  which 
had  a  similar  appearance  like  traffic-line  markings  and  which  were  virtually  laid  down  on 
the  road  surface  just  a  few  meters  in  front  of  the  road  crossing,  where  the  driver  had  to 
turn  left  or  right.  This  HUD  which  is  shown  on  figure  5  was  easy  to  interpret  by  the 
driver  but  couldn’t  be  read  from  a  distance  from  the  crossing.  As  a  result  of  these  studies 
both  of  the  two  displays  were  combined.  The  simple  HUD  was  used  to  indicate  that  the 
driver  has  to  turn  left  or  right  in  the  next  seconds  and  the  contact  analogue  HUD  was  used 
to  indicate  the  exact  position  where  he  had  to  leave  his  present  road. 
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Figure  5:  Contact  analogue  HUD  for  a  navigation  control  system. 
Conclusions 

Driving  simulators  are  not  in  the  first  place  a  tool  to  adjust  existing  HUD  concepts  to  a 
special  vehicle,  but  a  good  tool  to  examine  the  man-machine-interface  of  new  kinds  of 
HUDs  without  the  hmitations  of  acmal  technical  practicabilities.  By  doing  some  studies 
with  a  new  HUD  concept  in  a  simulator  the  expense  to  inte^ate  a  prototype  in  a  vehicle 
can  be  reduced,  because  it  is  possible  to  limit  the  indicated  information  to  those  parts  of 
the  concept  that  were  accepted  by  subjects. 


As  an  additional  field  for  simulator  studies  it  is  possible  to  examine  if  the  HUD  technique 
improves  the  security  in  different  traffic  situations.  For  example  it  would  be  possible  to 
evoke  a  special  event  sometimes  the  driver  reads  the  speedometer.  This  event  -  for 
example  a  ball  that  suddenly  rolls  on  the  road  -  should  cause  the  driver  to  reduce  his 
speed  immediately.  By  comparison  of  the  periods  of  times  needed  to  reduce  the  speed  by 
5  km/h  for  example,  or  by  comparison  of  the  part  of  events  where  an  accident  could  be 
avoided  an  objective  security  judgement  of  the  HUD  technique  in  vehicles  would  be 
possible.  In  order  to  measure  the  influence  of  the  HUD  technique  and  not  just  typical 
simulator  effects  a  kinesthetic  feedback  to  the  driver  is  absolutely  necessary  to  avoid  the 
loss  of  control  while  having  no  optical  feedback  about  the  actual  driving  situation.  To  be 
able  to  do  these  kind  of  studies  in  our  simulator,  too,  we’re  actually  examine  different 
possible  kinesthetic  feedback  systems  for  our  simulator. 
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Abstract 

We  are  developing  a  system  for  the  marker-free  analysis  of  hand  position  in  real-time, 
which  performs  in  undefined,  non-stationary  environments.  The  system  is  based  on  a 
frame-to-frame  SSD  correlation  of  greyscale  video  images,  and  currently  provides  the 
two-dimensional  coordinates  of  thumb,  index  fingertip  and  hand  at  a  frame-rate  of  14 
Hz.  This  system  can  be  applied  as  a  man-machine  interface(e.g.  replacing  the  mouse), 
and  for  the  recording  of  human  motor  performance  in  complex,  uncontrolled  scenarios 
(e.g.  clinics,  military,  aerospace). 

Introduction 

Several  systems  for  the  determination  of  3 -dimensional  limb  positions  are  available  on 
the  commercial  market.  They  all  require  that  „markers“  are  affixed  to  various  locations 
on  the  limb,  and  track  the  marker  positions  through  space.  Some  systems  employ  active 
markers  such  as  light  sources  (e.g.  Selspot®,  Optotrack®)  or  induction  coils  (Fastrak®), 
while  others  use  passive  markers  such  as  fluorescent  balls  (Macreflex®).  Unfortunately, 
however,  all  markers  obstruct  the  free  movement  of  the  limb,  due  to  their  physical  size, 
weight,  and  in  case  of  active  markers  the  leads  to  operate  them.  Thus,  markers  modify 
what  they  are  supposed  to  measure. 

As  a  second  problem,  marker-based  systems  require  special  environments  for  error-free 
operation,  such  as  rooms  free  of  metal  or  glossy  surfaces.  This  excludes  their  application 
in  many  real-life  scenarios,  where  environment  is  not  controlled. 

As  a  consequence,  we  decided  to  develop  a  marker-free  systems  which  can  operate  in 
undefined,  nonstationary  and  nonhomogenious  environments.  To  be  of  practical  use,  the 
system  should  have  a  spatial  accuracy  of  about  1  mm  and  a  temporal  resolution  of  100 
Hz.  Such  a  system  could  be  deployed  in  scenarios  such  as  virtual  reality,  man-machine 
interaction,  testing  of  human  performance  in  clinical,  military,  and  aerospace  settings,  or 
quality  control.  The  long-term  objective  is  to  achieve  3  dimensional  measurements  by 
means  of  2  cameras. 

Methods 

In  the  following  we  describe  the  actual  state  of  the  development  and  the  methods  used. 
Test  setup 

The  test  semp  of  this  system  is  shown  in  figure  1, 

Simple  graphical  objects  varying  in  shape  and  position  are  displayed  on  a  computer 
monitor.  The  subjects  task  is  to  point  onto  the  targets  with  his  index  finger  or  to  grasp 
them  with  his  index  finger  and  thumb  Fig  1 ,  These  movements  are  recorded  by  a  camera. 
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the  signal  of  which  is  digitised,  is  sent  to  the  CPU  of  a  PC,  and  evaluated  to  yield  2D 
hand  position  coordinates  in  real-time. 

The  working  range  for  the  hand  is  50  by  50  cm  with  a  distance  of  90  cm  between 
camera  and  screen. 

Hardware 

The  actual  hardware  used  is  one  camera  DALSA  CAD  1-256  with  8  bit  greyscale  and 
256*256  pixel  resolution,  a  lens  COSMICAR  8mm,  one  framegrabber  CORECO 
OCULUS  F64  (with  signal  processor  TMS  320C40  and  special  image  processing 
functions)  and  one  standard  Pentium  MMX  200  MHz  PC. 

Possible  ways  of  image  analysis 

Camera! 


Fig,  1  Test  setup. 

To  determine  hand  positions  from  the  recorded  frames,  we  could  use  several  techniques: 

1 .  Edge  detection  with  fixed  or  adaptive  threshold  level. 

2.  Filtering  with  different  kind  of  operators  (Laplace,  Sobel,  Prewitt ...)  and  subsequent 
edge  detection  with  fixed  or  adaptive  threshold  level. [1,3] 

3.  Knowledge  based  methods  which  fit  descriptive  models  direct  in  the  greyscale  or 
filtered  image.  [1,2,6,7,8,9,10] 

4.  Transformational  methods,  e.g.,  Gabor  filter,  FFT,  Hough  transformation  to  change 
from  the  spatial  space  into  a  transformation  space.  [1,2] 

5.  Neural  networks  which  work  on  greyscale,  filtered  images,  or  following  an 
transformation  method.[l  1] 

6.  Direct  pattern  searching  in  the  greyscale  image  with  pattern  matching 
algorithms.  [1,2, 4] 

Methods  1  and  2  are  highly  dependent  on  in  the  quality  of  the  image  background  and  the 
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structure  of  illumination  and  their  result  is  not  an  distinct  position.  Their  benefit  lies  in 
the  field  of  image  pre  processing,  and  augmented  by  simple  knowledge  based  rules. 

Methods  3,4  and  5  are  cumbersome  and  time  consuming,  and  may  therefore  be 
unsuitable  for  high  speed  real-time  applications. 

We  therefore  decided  to  use  method  6,  based  on  a  fast  pattern  matching  algorithm  similar 
to  theSSDalgorithms[4]. 


The  pattern  matching  method 

The  presented  method  is  based  on  a  combination  of  pattern  matching  and  knowledge 
based  evaluation  of  the  image.  A  modified  SSD  algorithm  is  used  with  dynamically 
updated  search  pattern  and  greyscale  discrimination  to  determine  the  required  positions. 
The  knowledge  based  component  is  used  to  determine  the  search  areas,  and  the  greyscale 
discrimination  parameters  to  refine  the  determined  finger  positions. 

To  start  the  analysis,  the  subject  has  to  bring  his  hand  in  a  pre  specified  starting  position 
in  front  of  the  camera.  The  software  then  determines  the  pixel  pattern  of  the  known 
locations  of  thumb,  index  finger  and  the  area  where  thumb  and  index  finger  meet  (which 
we  will  call  "hand'*  in  future)  as  shown  in  Fig.2.  these  patterns  serve  as  templates  for  the 
subsequent  search  algorithm.  The  size  of  these  templates  is  14*14  pixel  for  the  index 
finger  and  thumb,  and  22*40Pixel  for  the  hand.  The  system  determines  the  mean  grey- 
value  from  a  6x6  window  of  the  centre  of  the  hand  template. 

As  a  next  step,  the  pattern  matching  algorithm  searches  in  the  second  frame  for  a  pixel 
pattern  that  is  maximally  similar  to  that  of  the  hand  template  from  the  first  frame.  This 
search  is  limited  to  a  search  area  of  39x57  pixels  and  the  template  is  displaced  within 
this  area  in  horizontal  and  vertical  steps  of  3  pixels.  Once  a  best  position  is  found,  the 
search  is  refined  locally  displacing  the  template  in  steps  of  1  pixel. 

As  a  further  step,  the  algorithm  searches  in  the  second  frame  for  a  pixel  pattern  that 
resembles  the  thumb  and  index  finger  templates,  respectively,  within  a  search  area  of 
23x23  pixels.  ITie  templates  are  displaced  in  this  search  in  steps  of  one  pixel.  To  prevent 
that  the  found  pattern  "runs "  towards  the  hand  pattern  [5],  the  algorithm  searches  not  for 
the  maximum  similarity,  but  rather  for  a  location  as  far  as  possible  to  the  left  of  the 
maximum  (for  right  handed  subjects)  which  still  yields  at  least  90%  of  the  maximum 
value. 
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index  finger  pattern 


Thumb  pattern 


Fi^,  2  Search  templates. 

Once  the  hand  and  the  finger  pattern  are  found  in  the  second  frame,  they  are  used  as 
templates  in  the  third  frame,  etc.  in  an  iterative  procedure. 

To  determine  the  grade  of  similarity  between  a  template  and  a  pixel  pattern  in  a  image, 
we  use  the  modified  SSD  algorithm[4]  shown  in  function  (1).  This  function  gives  a  value 
MR(sx,sy)  that  becomes  0  at  the  location  sx,sy  in  the  image  with  best  coincidence 
between  template  and  the  pixel  pattern.  The  factor  k  provides  the  greyscale 
discrimination.  If  the  grey  value  of  one  pixel  at  x,y  in  the  template  or  the  corresponding 
position  x+sx,y+sy  in  the  image  lies  not  within  ±  25  %  of  the  mean  grey  value,  k 
becomes  0,  else  k  is  set  to  1.  Nx  and  Ny  represent  the  size  of  the  template  N  =  Nx*Ny  is 
the  normalisation  faktor,  with  each  k  =  1  N  becomes  decreased  by  1  to  ensure  a  proper 
normalisation,  Ixy  and  P^y  are  the  grey  values  of  the  template  and  image  respectively.  The 
result  of  the  search  Function  (1)  is  shown  in  Fig.  3  for  a  large  search  area  and  an  index 
fingertip  as  search  template. 


(1) 


Nx  Ny,  , 

L  £  k*(Px,y- 
x=0y=0 


MR(sx,sy)  = 


x+sx,y+sy 


N 
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Old  handposition  Atx  \ 

New  handposition 

Fig  4  Determination  of  finger  search  areas. 

Further  features  of  the  algorithm 

In  order  to  increase  the  speed  of  the  search  algorithm,  we  displace  the  finger  search  areas 
in  each  frame  along  with  the  displacement  of  the  handposition.  To  achieve  this  we  use 
the  relative  distances  of  the  finger  positions  to  the  hand  position  (Afx,Afy)  and  (Atx,Aty) 
in  the  previous  frame  and  use  these  with  the  hand  position  in  the  actual  frame  to 
determine  the  search  area  for  the  finger  positions  Fig  4.  So  we  could  reduce  the  finger 
search  area  shift  to  23  pixels  in  x  and  y  direction.  Due  to  the  fact  that  the  template  and 
the  search  area  are  now  overlapping  with  about  50%  of  template  size,  the  algorithm 
becomes  more  robust  against  locations  in  the  image  background  with  a  high  grade  of 
coincidence  to  the  searched  template.. 

Results 

With  the  system  presented  in  this  paper,  we  achieve  an  recognition  rate  of  15  frames  per 


second,  or  67  ms  per  frame,  yielding  3  coordinates  per  frame.  The  frame  time  consists  of 
20  ms  for  image  analysis,  and  47  ms  for  transferring  the  images  from  the  frame  grabber 
to  the  PC. 

Hand  and  finger  positions  are  tracked  successfully  and  independent  from  image 
background,  while  the  hand  executes  transitional  or  rotational  movements,  or  changes  its 
shape. 

The  accuracy  of  tracking  the  finger  positions  is  ±  3  mm  from  frame  to  frame,  and  ±  6 
mm  across  30  frames. 

The  maximal  trackable  actual  velocity  is  ±  25  cm  per  sec  for  the  hand,  and  ±  12  cm  per 
sec  for  the  finger. 

With  an  increase  of  the  recognition  rate  to  100  frames  per  second  the  maximal  trackable 
velocities  should  increase  to  ±  166  cm  per  sec  for  the  hand  and  ±  80cm  per  sec  for  the 
finger.  These  values  correspond  to  very  fast  real  life  hand  movements. 

Next  steps 

Our  future  work  will  address  the  following  issues. 

•  Improvement  of  accuracy  for  the  finger  positions  by  improvement  of  the  search 
technique  which  actually  determines  the  leftmost  finger  position  (for  right  handed 
subjects). 

•  Resolving  problems  with  overlapping  fingers  by  adding  shape- orientated  components 
into  the  matching  algorithm 

•  Increase  of  processing  speed  by  migrating  and  parallelisation  of  the  algorithm  from 
the  PC  to  the  signal  processor  and  the  coprocessing  system,  and  by  reducing  the 
search  areas  by  means  of  a  prediction  algorithm  for  the  next  finger  and  hand  position. 

•  Introduction  of  three  dimensional  processing  with  2  cameras  to  achieve  three 
dimensional  coordinates.  The  extension  of  the  system  to  two  cameras  will  provide  real 
3  dimensional  coordinates  and  the  possibility  to  dynamically  adapt  the  size  of  search 
templates  and  search  areas.  While  the  algorithm  gives  only  2-D  coordinates  it  is  a 
relative  simple  task  to  transform  the  two  camera  outputs.  This  task  can  be  simplified 
by  a  dextrous  camera  arrangement. 

•  Automatic  determination  of  the  positions  in  the  very  first  frame  by  means  of  neural 
networks  which  recognise  the  hand  shape  irrespective  of  their  position.  This  approach 
which  has  recently  started  in  our  lab,  will  eliminate  the  need  pre-specified  starting 
position. 

Conclusion 

The  presented  preliminary  system  provides  a  encouragingly  fast  and  accurate,  marker- 
free  algorithm  to  determine  the  position  of  3  interconnected  objects  with  almost  no 
constraints  on  image  background  or  light  conditions.  Greyscale  cameras  are  sufficient 
for  our  system  and  faster  in  processing  than  colour  cameras  would  be.  The  algorithm  is 
applicable  to  objects  with  a  narrow  banded  histogram.  The  field  of  application  is  not 
limited  to  physiological  applications  and  rather  could  include,  e.g.,  the  localisation  of 
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any  moving  objects  in  an  unstructured  environment.  The  system  can  be  used  as  a 

interface  in  virtual  environment  generators. 
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Abstract 

The  centralised  control  rooms  of  large  industrial  plants  have  separated  people  from  the 
processes  they  should  control.  Perception  is  restricted  mainly  to  the  visual  sense.  Only 
telephone  or  radio  links  provide  narrow-band  voice  communication  with  maintenance 
personnel  down  in  the  plant.  Multimedia  equipment  can  perceptionally  bring  back  the 
operator  into  the  plant  while  bodily  keeping  him  the  comfortable  and  safe  control  room. 
TTiis  involves  video  and  audio  transmission  from  process  components  as  well  as  sights 
and  sounds  artificially  generated  from  measurements.  Groupware  systems  support  inter¬ 
action  between  operators,  engineers,  and  managers  in  different  plants.  With  support  from 
the  German  government,  the  state  of  Hessen,  and  industrial  companies  the  Laboratory  for 
Systems  Engineering  and  Human-Machine  Systems  at  the  University  of  Kassel  estab¬ 
lishes  an  Experimental  Multimedia  Process  Control  Room.  Core  of  this  set-up  are  two 
high-performance  graphics  workstations  linked  to  one  of  several  process  or  vehicle 
simulators.  Multimedia  periphery  includes  video  and  teleconferencing  equipment  and  a 
vibration  and  sound  generation  system. 

Introduction 

Today,  operators  in  few  or  even  only  one  central  control  room  are  separated  from  the 
process  control  large  industrial  plants.  Increasingly  sophisticated  control  systems  shift  the 
role  of  the  human  operator  from  manual  to  supervisory  control,  and  research  for  autono¬ 
mous  operation  of  large  plants  is  in  progress.  However,  the  presence  of  the  operator  in  the 
control  room  of  a  plant  is  still  necessary.  Thus,  the  interface  to  the  process  and  its  control 
system  remains  an  important  component.  Process  operators,  seeing  only  selected  physical 
measurements  of  large  systems  on  their  small  displays,  and  physically  separated  from  the 
equipment  loose  contact  to  the  process. 

Before  introduction  of  CRT  technology,  process  control  had  walls  showing  mimic  dia¬ 
grams,  electro-mechanical  instruments,  and  mechanical  switches.  Still  today,  mimic  dia¬ 
grams  and  instrument  walls  influence  the  design  of  CRT  displays.  While  the  number  of 
process  variables  increased,  the  task  of  deriving  the  necessaiy  information  about  the  pro¬ 
cess  state  out  of  all  these  variables  is  still  mainly  left  to  the  operator.  The  operator  has  to 
perceive  a  vast  amount  of  information.  Except  for  acoustical  alarms,  the  visual  channel 
has  to  convey  the  majority  of  this  information  while  conventional  interfaces  do  not  yet 
take  advantage  of  other  human  senses  and  new,  multimedia-based  technologies. 

Experience  in  nuclear  power  industry  shows,  that  plant  operators  do  not  need  support 
systems  during  95%  to  98%  of  operation.  In  the  remaining  cases,  support  systems  do  not 
always  provide  the  support  necessary  and  the  operators  depend  on  their  own  knowledge. 
Although  highly  trained  and  motivated,  operators  are  unable  to  cope  with  situations  when 
they  cannot  access  the  necessary  information  about  system  state  and  performance.  In  the 
2%  of  operational  cases,  that  are  unusual,  complex,  and  mostly  new,  the  operator  needs 
all  available  information  to  solve  the  problems.  This  information  may  comprise  phenom- 
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ena  not  transferred  by  usual  measurements  and  unsuited  to  conventional  displa^ys.  Addi¬ 
tional  media,  integrated  in  the  control  room,  may  be  able  to  convey  the  information. 

Extensive  task  and  knowledge  analyses  performed  by  our  laboratory  in  a  variety  of  in¬ 
dustrial  processes  showed  that  an  experienced  operator  uses  more  information  from  a 
plant  than  just  the  measurements  displayed  in  the  control  room.  The  noise  spectrum, 
sounds,  vibration,  temperatures,  direct  vision,  even  odour  adds  information  to  form  the 
overall  picture.  It  may  be  helpful  to  bring  this  information  back,  specially  to  the  operator 
who  is  more  and  more  separated  (physically  as  well  as  concepmally)  from  the  process. 
Opposed  to  hardware  and  software  used  in  process  control  rooms  today,  equipment  and 
procedures  used  in  computer  graphics  show  immense  improvements.  This  offers  the  gen¬ 
eration  of  virtual  images  based  on  physical  measurements,  abstract  data,  and  process 
structure.  Multimedia  technology  can  bring  the  process  closer  to  the  operator  in  a  distant 
control  room  by  adding  some  of  the  lost  sensory  channels.  Instead  of  showing  the  icon  of 
a  tank  besides  a  level  measurement,  the  computer-generated  image  can  show  the  tank  it¬ 
self,  along  with  the  correct  liquid  level  inside  and  a  sound  depicting  liquid  flow.  Using 
computer  graphics  and  virtual  reality  algorithms,  other  measurements  can  enhance  the 
image,  for  example  component  colour  for  temperature,  surface  structure  for  efficiency. 
Using  an  image  of  a  process  component  instead  of  physical  measurements,  showing  live 
videos  along  with  stereo  sound  from  the  process  can  again  establish  the  close  contact  to 
the  process.  Simulating  a  reality  sufficiently  close  to  the  operator’s  view  of  the  process 
will  enable  the  operator  to  identify  problems  quickly.  A  live  video  image  can  show  proc¬ 
ess  failures  not  captured  by  measurements.  In  parallel,  video  conferencing  enhances  con¬ 
tact  between  process  control  operators  and  maintenance  personnel  in  the  plant.  Beyond 
this,  groupware  and  co-operative  interfaces  for  multi-user  interaction  in  plant-wide  con¬ 
trol  and  communication  can  support  problem  solving  on  aU  levels,  such  as  the  opera¬ 
tional,  the  maintenance,  or  the  management  level. 

We  know  from  own  experience,  how  much  the  background  sound  in  a  control  room  next 
to  the  turbine  room  and  boiler  changes  with  slight  variations  in  the  plant  state.  Experi¬ 
enced  operators  will  benefit  from  good  reproduction  of  these  phenomena.  Besides  giving 
back  information  to  the  operators  they  had  in  conventional  systems  it  is  also  possible  to 
use  these  additional  media  to  transmit  new  information  and  to  support  collaborative  work. 

The  Future  Control  Room:  Co-Operative  Work  and  Multimedia  Support 
Because  of  the  growing  complexity  of  systems,  more  operators,  even  in  different  loca¬ 
tions  in  the  plant  need  to  co-operate.  Telephone  or  radio  communication  provides  only 
poor  means  to  exchange  process  measurements  and  drawings  from  operation  manuals 
along  with  views  and  ideas  for  co-operative  problem  solving.  The  co-operation  between 
different  human  user  classes  (e.g.,  operators,  engineers  and  managers)  can  be  facilitated 
by  the  combination  of  several  multimedia  representations  on  different  abstraction  levels. 
The  different  information  needs  of  these  user  classes  are  considered  in  the  multi-human 
interface  design  by  providing  them  with  dedicated  windows  or  screens  and  audio  infor¬ 
mation.  Engineers  and  managers  may  more  often  want  to  use  goals-means  hierarchies  and 
multilevel  flow  model  presentations  whereas  operators  can  use  the  less  abstract  presenta¬ 
tions  of  ecological  and  topological  displays.  However,  free  navigational  access  to  all  dis¬ 
play  options  must  be  allowed  for  all  user  classes,  based  on  the  different  focuses  of  their 
individual  preferences.  The  concepts  of  visual  momentum  and  cognitive  layouts  should 
be  implemented  in  such  a  way  that  they  support  the  integrated  view  among  team  mem- 
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bers  of  multi-user  groups  across  all  different  forms  of  graphical  representations.  Related 
video  and  audio  information  further  support  this. 

A  large  number  of  pictures  in  a  complex  display  network  is  characteristic  for  many  in¬ 
dustrial  applications.  In  such  cases,  only  some  of  these  pictures  may  be  individualised  for 
different  user  classes.  Tani  et  al.  (1994)  suggested  to  use  a  large  screen  as  a  shared  dis¬ 
play  for  all  group  members,  together  with  detailed  personal  displays  on  individual  work¬ 
station  screens  for  each  of  the  group  members.  Multimedia  presentations  of  live  video 
can  be  combined  with  computer  graphics  presentations.  The  visual  momentum  is  imple¬ 
mented  by  means  of  highlighting  manipulated  objects  as  well  as  by  corresponding 
movements  of  individual  cursors  for  each  group  member  in  the  shared  and  detailed 
screens. 

Relatively  new  communication  technologies  for  human-machine  interaction  have  been 
developed  with  the  field  of  multimedia.  The  main  idea  of  multimedia  communication  is 
to  combine  and  integrate  different  visual  and  auditory  media  for  the  display  and  visuali¬ 
sation  of  information  about  tasks  to  be  performed  with  a  machine  or  a  computer,  hi  par¬ 
ticular,  the  following  media  are  combined  v^th  each  other:  computer-generated  visual 
displays,  video  recordings,  and  auditory  information  such  as  recordings  of  noises  and 
synthetic  speech  (Steinmetz  and  Herrtwich,  1991).  In  addition,  three-dimensional  stereo¬ 
scopic  video  scenes  and  stereoscopic  computer  graphics  can  be  superimposed  upon  one 
another  (Milgram,  et  al,  1990).  Even  music  and  haptic  information  as  well  as  smell  may 
be  used  for  multimedia  communication. 

A  connection  of  multimedia  objects  with  an  information  network  can  be  used  interac¬ 
tively,  for  example,  in  travel  agencies  and  libraries  (Jerke,  et  al.  1990),  In  this  pure  hu¬ 
man-computer  interaction,  the  querying  technique  is  combined  with  the  method  of 
browsing  in  a  hypertext  environment.  A  hypertext  system  is  a  network  of  information 
nodes  that  are  connected  by  links,  in  a  non- sequential  way,  to  arbitrary  non-hnear  infor¬ 
mation  stmctures  (Bogaschewsky,  1992).  A  hypertext  network  has  been  combined  with 
an  automatic  fault-diagnosis  module  in  order  to  support  the  problem-solving  activities 
during  trouble-shooting  (particularly,  motor  diagnosis)  tasks  of  human  technicians  in  car- 
repair  shops  (Hollender,  1995). 

In  general,  the  human  user  can  navigate  through  a  hypertext  network  by  analysing  the 
connections  between  the  nodes.  Conceptual  connections  between  alternative  interaction 
procedures  lead  to  a  hypermedia  information  network.  The  separate  nodes  of  this  network 
are  related  to  the  separate  types  of  information,  such  as  text,  graphics,  video  or  audio, 
which  are  different  forms  of  multimedia.  Thus,  hypermedia  represent  an  information 
concept  connecting  several  media  (Begoray,  1 990),  whereas  multimedia  encompass  a 
combination  of  different  presentation  media.  The  terms  hypermedia  and  multimedia 
are,  thus,  distinguishable  from  each  other  in  a  similar  way  as  the  dialogue  and  the  pres¬ 
entation  layers  are,  in  a  user  interface  management  system. 

Multimedia  communication  systems  will  be  introduced  in  the  future  in  several  application 
domains.  We  mentioned  libraries  and  travel  agencies  already  above.  In  addition,  the  en¬ 
tertainment  industry  is  highly  interested  in  this  new  technology.  Particular  applications 
will  be  possible  in  medicine,  for  co-operative  conferencing  between  physicians  (Klein- 
holz  and  Ohly,  1994)  and  for  surgery.  Industrial  applications  may  be  possible  in  glass 
production,  the  paper  and  pulp  industries,  the  chemical  industries  (Heuer,  et  al.,  1994), 
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the  power  industry  (Zinser,  1993),  and  the  maintenance  of  networks  such  as  those  for  dis¬ 
tribution  of  electricity  (Akiyoshi  et  al,,  1995).  Tanaka  et  al.  (1988)  suggested  a  tutoring 
system  for  the  latter  case. 

Alty  and  Bergan  (1992)  emphasise  that  many  questions  still  exist  with  respect  to  the  ap¬ 
plication  of  multimedia  communication  in  dynamic  industrial  process-control  tasks.  The 
general  problem  of  the  interface  design  remains:  Which  information  is  needed  by  the 
human  user,  when,  in  which  form  and  why?  This  problem  becomes  more  severe  with  a 
larger  number  of  technological  options,  which  increase  further  with  the  multimedia  do¬ 
main.  Therefore,  the  information  needs  of  the  later  human  users  have  to  be  investigated 
by  means  of  task  analyses.  This  requirement  becomes  even  more  important  in  the  cases  of 
co-operation  among  several  human  users  in  co-operative  work  situations. 

Expert  analyses  (performed  in  a  cement  plant,  as  a  kind  of  unstructured  task  analysis)  in¬ 
dicated  some  aspects  of  the  co-operative  work  situations  in  that  particular  application  do¬ 
main.  The  face-to-face  communication  is  absolutely  mandatory.  In  addition,  the  audio 
channel,  e.g.,  telephone  communication,  is  very  important.  Multimedia  technologies  can 
be  used  for  integrating  the  video  information  from  some  of  the  equipment,  which  is  now 
available  in  the  control  room  on  separate  video  screens.  Otherwise,  video  observations 
are  rejected  as  a  spy  system  in  this  application  domain.  Consequently,  the  important 
aspects  of  multimedia  communication  for  cement  planp  comprise:  (1)  computer  graphics 
and  related  video  presentations  with  interaction  facilities  for  human-machine  and  human- 
human  communication,  as  well  as  (2)  teleconferencing  with  highly  improved  face-to-face 
video  presentations  and  screen-based  audio  communication.  Both  aspects  need  to  be  re¬ 
lated  to  one  another  and  integrated  in  a  task-oriented  manner.  Overlapping  windows  need 
to  be  avoided  as  much  as  possible,  as  usual  in  process  control  applications. 

The  important  aspects  for  travel  agencies  do  not  include  the  teleconferencing  part.  How¬ 
ever,  a  much  more  substantial  search  for  related  multimedia  objects  in  an  information 
network  is  required  (Jerke,  et  al,  1990).  The  human  user  classes  are  travel  agents  and 
customers,  who  are  possibly  also  co-operating  with  one  another  for  more  efficient  cus¬ 
tomer  support.  Special  computer  skills  cannot  be  expected  from  the  average  customer. 
This  leads  to  the  necessity  of  easy-to-use  multimedia  interfaces  with  related  text,  picture, 
graphics,  video  and  audio  information. 

The  same  general  tendency  exists  with  the  important  aspects  of  multimedia  communica¬ 
tion  in  the  entertainment  sector.  Everybody  wants  to  participate  and  enjoy  in  an  easy,  di¬ 
rect  and  interactive  way,  comparable  to  strolling  through  a  garden  or  singing  a  song,  e.g., 
in  a  Karaoke  environment  (Tamura,  1995).  This  most  intuitive  computer  use  required  in 
the  entertainment  domain  will  probably  contribute  strongly  to  new  powerful  and  highly 
user-friendly  multimedia  systems.  Although  they  will  appear  more  as  game  instruments 
rather  than  as  conventional  computers,  they  may  become  the  next  generation  of  multime¬ 
dia  computers  also  in  industrial  and  service  applications. 

The  research  into  computer  supported  co-operative  work  (CSCW,  1994)  and  groupware 
deals  with  theories,  design  concepts,  architectures,  prototype  systems,  and  empirical  re¬ 
sults  for  co-operative  work  situations  in  application  domains  such  as  offices,  classrooms 
and  factories.  Electronic  mail  systems,  computer  and  videoconference  systems,  office  in¬ 
formation  systems,  organisational  knowledge  bases,  shared  window  systems  and  other 
communication  and  co-operation  systems  are  being  investigated.  Some  of  these  systems 
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do  not  allow  the  human  users  to  interact  or  co-operate  with  each  other  directly  in  space 
and  time.  New  approaches  for  the  integration  of  action  space  and  time  have  been  sug¬ 
gested,  e.g.,  by  Ishii,  et  al  (1992).  Such  support  for  co-operative  work  seems  to  be  par¬ 
ticularly  required  in  industrial  human-machine  systems. 

As  the  results  from  the  expert  analyses  in  the  cement  plant  indicate,  co-operative  work 
should  be  organised  as  far  as  possible  using  face-to-face  communication.  Large  projec¬ 
tion  screens  are  not  welcome,  because  they  are  very  soon  too  overloaded,  and  are  not 
adaptable  enough.  However,  they  have  already  been  implemented  in  some  other  applica¬ 
tion  domains,  but  the  concept  of  overlapping  information  for  different  human  user  classes 
has  not  been  taken  into  account  in  the  information  content  of  these  projection  screens. 
Thus,  their  use  for  co-operative  work  needs  to  be  rethought.  Display  screens  for  group 
meetings  in  different  offices  and  the  control  room  are  welcomed  as  multi-human  machine 
interfaces  in  the  cement  plant.  They  will  also  be  accepted  as  dedicated  human-machine 
interfaces  in  a  network,  and  for  discussions  of  smaller  problems  over  the  telephone. 

The  display  screens  for  the  group  meetings  may  consist  of  one  screen  with  four  to  five 
windows.  They  allow  access  to  all  pictures  in  the  control  room,  rather  than  having  just 
printouts,  as  presently  available.  Different,  most  favoured  pictures  may  be  selected  by 
different  user-group  representatives.  All  the  selected  pictures  need  to  be  seriously  consid¬ 
ered  by  all  members  of  the  meeting,  because  co-operation  rather  than  ego-centred  views 
are  required,  where  each  user-group  representative  contributes.  Modifications  of  control- 
room  pictures  are  foreseeable  for  the  display  screens  in  group  meetings.  Quick-change 
and  easy-to-use  editing  facilities  may  allow  the  selection  of  important  lines  or  variables 
from  a  table,  qualitative  zooming-in  and  selection  of  subareas  of  component  flow  dia¬ 
grams,  and  manoeuvring  or  selection  by  sliders  or  text  menus  through  different  levels  of 
abstraction.  The  latter  range  from  physical  forms,  such  as  scanned-in  photos,  e.g.,  taken 
from  databases  or  just  of  broken  components  (inside  a  pump,  etc.),  to  goal  hierarchies  via 
multilevel  flow-modelling  representations. 

The  consistency  and  coherence  across  selected  and  edited  multimedia  has  to  be  guaran¬ 
teed,  This  will  support  the  visual  momentum,  which  is  now  already  available,  when  a 
trend  curve  is  selected  by  the  cursor  for  a  particular  variable  in  the  component  flow  dia¬ 
gram.  The  consistency  will  be  increased  when  the  computer  completes  the  other  selected 
pictures  shown  in  parallel,  e.g.,  consistent  with  the  information  reduction  in  the  just- 
edited  picture.  The  information  filtering,  reduction  and  qualitative  modifications  may  be 
supported  by  the  computer,  or  can  be  done  solely  by  the  group  members.  Further,  com¬ 
puter-supported  drawing  facilities,  e.g.,  for  straight  lines  or  for  rapid  prototyping 
(sketching)  of  new  pictures  and  ideas  or  for  modifying  existing  ones,  are  possible.  How¬ 
ever,  they  may  be  more  suitable  for  exploratory  purposes  rather  than  for  normal  group 
meetings,  because  the  latter  might  become  too  long  with  too  much  computer  interaction. 

The  overlapping  information  for  the  different  user  classes  has  already  been  considered  in 
the  logbook,  now  available  on  a  PC  in  the  cement  plant.  This  information  has  still  to  be 
implemented  further  in  the  presentation,  the  dialogue  and  the  explanation  facilities  of  the 
human-machine  interfaces,  particularly  of  the  display  screen  for  group  meetings. 
Thereby,  the  visual  momentum  between  different  windows,  which  relate  primarily  to  dif¬ 
ferent  user-group  representatives,  has  to  be  supported. 

All  the  suggested  designs  of  human-machine  interfaces  for  co-operative  work  also  have 
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to  consider  face-to-face  communication.  Otherwise,  the  social  contacts  would  not  be  im¬ 
proved  if  this  face-to-face  contact  were  to  disappear.  Tele-co-operation  is  often  not  feasi¬ 
ble  because  the  contact  with  the  production  will  be  lost,  e.g.,  the  feeling  for  clinker  qual¬ 
ity  will  disappear.  In  addition,  the  work  climate  will  deteriorate  and,  thus,  there  will 
eventually  be  no  co-operation. 

Design  work  is  often  accomplished  in  teams  where  human  users  interact  and  co-operate 
with  each  other  directly  in  space  and  time.  New  approaches  for  the  integration  of  action 
space  and  time  have  been  suggested,  e.g.,  by  Ishii  et  al.  (1994),  as  mentioned  above.  Two 
people  work  on  the  same  drawing  for  design  and,  at  the  same  time,  see  the  co-operative 
design  partner  through  the  transparent  digitiser  sheet.  The  two  designers  can  physically 
work  in  remote  places  from  each  other.  The  basic  metaphor  of  this  concept  is  'Talking 
through  and  drawing  on  a  big  transparent  glass  board.”  Tlius,  it  is  believed  to  be  very  im¬ 
portant  to  have  face-to-face  conununication  as  well  as  audio  communication  with  each 
other  in  a  team. 

Similar  design  principles  were  used  for  a  co-operative  work  support  system,  which  allows 
the  supervisory  control  as  well  as  the  maintenance  of  a  power  plant  (Muraoka,  Ohi, 
1995).  Not  only  image  data  but  also  audio  and  drawing  data  are  combined  with  each 
other.  Several  windows  on  the  display  screen  allow  to  visualise  in  parallel  topological 
component-oriented  presentations  and  video  scenes  from  the  plant  or,  alternatively,  from 
the  face  of  the  communication  partner,  e.g.,  of  the  maintenance  person  in  the  field  work. 
Additionally,  a  communication  board  can  be  shown  in  a  further  window.  The  communi¬ 
cation  board  can  be  used  as  a  white  board  or  a  pin  board  on  which  selected  information, 
cut  out  from  any  other  window,  can  be  pasted.  This  white  board  is  shown  to  both  com¬ 
munication  partners  in  the  remote  sites.  They  can  circle  and  mark  or  write  anything  they 
wish  on  top  or  besides  the  pasted  information,  again  visible  also  for  the  remote  site.  Thus, 
a  direct  telecommunication  can  be  used  for  co-operative  problem  solving. 

The  already  mentioned  multimedia  means  of  video  conferencing  with  audio  and  face-to- 
face  communication  are  even  more  important  on  the  management  and  the  marketing  lev¬ 
els,  As  the  facial  communication  is  regarded  as  very  important  in  many  cultures,  it  seems 
to  be  worthwhile  trying  to  reduce  the  huge  amounts  of  data  to  be  transferred  in  telecom¬ 
munication.  It  is  feasible  to  simulate  computer  images  of  a  person  in  the  remote  partner’s 
place.  Then,  only  minimal  parameter  sets,  which  characterise  the  perceived  impressions 
of  a  human  face,  e.g.,  fierceness  or  gentleness  of  a  face,  have  to  be  transferred.  An  exam¬ 
ple  of  such  investigations  of  facial  features  was  presented  by  Kato  et  al.  (1995), 

Advances  in  virtual  reality  technology  allow  to  explore  freely  the  virtual  environment, 
e.g.,  of  a  maintenance  situation  by  manipulating  3D  graphical  objects.  Liquid-crystal 
glasses  and  data  gloves  have  to  be  used  for  viewing  and  manipulating  the  stereoscopic 
display.  Currently,  the  possibilities  for  exploiting  such  virtual  reality  and  multimedia 
technologies  for  learning  and  training  environment  and  for  marketing  are  investigated 
(Akiyoshi  et  al,,  1995). 

Integrating  the  above  approaches  across  aU  levels  in  an  appropriate  human  user-  and  task- 
oriented  manner  will  be  a  major  challenge  of  the  next  coming  years.  It  will  be  important 
to  think  about  desired  work  organisations,  first,  and  to  pursue  corresponding  cognitive 
task  analyses  in  order,  then,  to  build  plant-wide  control  and  communication  systems 
which  will  be  welcomed  with  high  user  acceptance.  It  is  also  very  likely  that  remarkable 
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cultural  and  socio-political  differences  with  respect  to  optimal  or,  at  least,  satisfying  so¬ 
lutions  may  exist  even  within  individual  countries  between  different  companies. 

Even  in  the  next  future,  process  control  rooms  will  need  the  presence  of  human  operators. 
Those  cases,  when  pre-programmed  automatic  systems  would  fail  to  control  the  process 
specially  demand  for  sldll  and  knowledge  of  the  operators.  In  these  rare  cases,  aU  avail¬ 
able  information  on  the  process  and  the  combined  skills  and  knowledge  of  operators, 
technicians,  engineers,  and  managers  is  needed  for  co-operative  problem  solving.  The 
integration  of  new  media  for  information  presentation  and  co-operative  work  to  support 
humans  in  these  situations  is  a  main  research  topic  of  the  new  Multimedia  Process  Con¬ 
trol  Room  established  in  the  Laboratory  for  Human-Machine  Systems  of  the  University 
of  Kassel,  The  next  sections  describe  the  desirable  capabilities  of  such  an  experimental 
instrument  and  the  current  implementation. 

Essential  Capabilities  of  a  Multimedia  Process  Control  Room 

For  technical  (and  financial)  reasons,  the  multimedia  process  control  room  in  its  current 
configuration  addresses  the  auditory  and  the  visual  sense  with  some  support  for  percep¬ 
tion  of  vibrations.  Visual  and  acoustic  information  is  presented  supporting  perception  of 
directions  and  depth.  Artificial,  fully  computer-generated  information,  play-back  of  pre¬ 
recorded  information,  as  well  as  life  presentation  is  foreseen.  In  detail,  a  multimedia  con¬ 
trol  room  should  provide 

•  Computer-generated  and  animated  visual  displays,  showing  process  information  and 
incorporating  textual,  pictorial,  and  graphical  objects,  combined  with  Hve  or  pre¬ 
recorded  video; 

•  life  video  image  display  for  plant  supervision  as  well  as  visual  inspection  of  equip¬ 
ment  or  products  and  video  presentations  for  operator  and  maintenance  support; 

•  life,  reproduced,  or  computer-generated  audio,  vibration,  and  speech  for  supporting 
visual  perception,  alerting,  and  informing  operators; 

•  3D  stereoscopic  and  holographic  presentation  of  the  generated  visual  scene  and  spa¬ 
tial  presentation  of  audio  information; 

•  face-to-face  video  communication  and  conferencing,  incorporating  document  ex¬ 
change,  drawing  and  annotation  facilities;  screen-operated  telephone  and  radio  com¬ 
munication;  and, 

•  maybe  in  the  future  haptic  information,  motion,  music,  odour,  and  more. 

For  research  and  experimentation  in  this  area,  an  experimental  set-up  needs  in  addition 
facilities  for  development  of  software,  scenes  and  environments.  Some  additional  prepa¬ 
rations  necessary  are 

•  recording  equipment  to  take  audio  and  video  samples  in  existing  plants,  import  filters 
for  CAD  data  from  existing  environments; 

•  modelling  software  to  generate  3D  objects  and  assembling  those  to  scenes  together 
with  audio  and  video  objects; 

•  various  means  for  manual  interaction  like  mouse,  joystick,  keyboard,  together  with 
related  analogue  and  digital  input  channels; 

•  software  to  generate  realistic  behaviour  of  the  interface,  like  plant,  process,  and  com¬ 
munication  simulators;  and 

•  support  for  supervision  and  control  of  experiments,  data  collection,  and  evaluation. 

The  following  section  describes  some  functions  of  a  multimedia  process  control  room 
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with  examples  assuming  a  live  process  environment. 

Computer  Graphics  and  Video  Support 

Today,  computer  graphics  in  process  control  rooms  are  still  too  much  copying  the  tradi¬ 
tional  means  of  walls  with  control  instruments  to  the  computer  screen,  while  separate 
monitors  provide  video  information.  Current  research  shows  the  value  of  displays  devel¬ 
oped  into  two  directions.  One  is  creating  more  abstract  views  that  are  appropriate  to  the 
operators’  task,  behaviour,  and  knowledge  (Ali,  1997;  Ali  and  Heuer,  1995;  Johannsen  et. 
al,  1997).  The  other  direction  supports  the  objective  to  bring  back  the  operator  into  tiie 
plant  by  generating  views  close  to  reality  (Wittenberg,  1997).  hi  both  cases,  integrating 
video  information  can  support  the  view. 

The  demand  for  further  enhancement  of  a  computer  generated  view  leads  to  stereoscopic 
and  holographic  displays.  Stereoscopic  displays  extend  the  view  from  the  flat  screen  into 
the  third  dimension  without  taking  account  of  the  observer’s  position.  The  common  tech¬ 
nology  generates  separate  images  for  each  eye  of  the  observer,  displayed  alternately  on 
one  screen  or  in  parallel  on  two  screens.  Head-mounted  devices  providing  one  screen  for 
each  eye  support  the  second  solution  while  the  observer  needs  to  wear  special  glasses  for 
the  first  version. 

Stereoscopic  views  look  abnormal  to  the  observer  when  moving  around  the  scene,  as  a 
3D  stereo  view  presents  the  image  seen  from  one  specific  position  only.  The  advantage  is 
that  the  image  is  the  same  for  every  viewer  fi:om  every  position.  The  disadvantage  is  that 
the  viewer  cannot  just  walk  closer  or  around  the  image.  Even  shifting  the  head  from  side 
to  side  or  forward-backward  does  not  change  the  view  as  expected  from  natural  scenes 
and  creates  an  annoying  effect. 

Holographic  views  take  into  account  the  observer’s  eye-point  by  tracking  the  position  of 
the  3D-glasses.  A  holographic  projection  adapts  to  shifts  in  all  directions  (left-right,  up- 
down,  to-from)  within  the  range  of  the  tracking  device.  The  observer  can  see  the  scene 
from  different  directions  as  long  as  he  looks  towards  the  screen  surface.  This  feature  eas¬ 
ily  adds  to  stereoscopic  views  with  additional  tracking  hardware  and  software  compo¬ 
nents.  The  display  generation,  however,  supports  only  the  view  of  one  person;  other 
persons  need  separate  sets  of  display,  glasses  and  tracker.  Using  head-mounted  screens 
and  sensing  position  and  orientation  provides  means  for  full  virtual  reality  that  also  shows 
views  from  the  reverse  side.  The  drawback  is  the  weight  of  the  helmet  to  carry  and  the 
isolation  from  the  real  world.  The  latter  also  acoustically,  as  common  headsets  moreover 
provide  headphones.  With  this  set  is  possible  to  walk  around  the  display  in  the  virtual 
world  but  difficult  to  avoid  at  the  same  time  falling  over  the  objects  in  the  real  world. 

Besides  video  integration  to  enhance  personal  communication  and  conferencing,  we  see 
the  necessity  of  two  different  video  applications  in  process  control:  Live  video  for 
equipment  or  product  observation,  and  video  sequences  for  operator  support.  Live  images 
from  the  plant,  integrated  into  computer  generated  diagrams  enable  operators  to  inspect 
critical  equipment  visually.  Cameras  either  placed  in  strategic  position  down  the  plant  or 
hand-held  (or  even  helmet-mounted)  by  maintenance  personnel  may  provide  these  sig¬ 
nals.  In  case  of  failures,  video  clips  can  instruct  operators  as  well  as  maintenance  person¬ 
nel  on  correct  procedures. 
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Audio  Support 

In  a  conventional  control  room,  the  audio  channel  provides  the  operator  mainly  with 
alarms.  A  multimedia  environment  adds  acoustic  information  from  the  process  as  sounds 
and  noises  that  provide  the  operator  with  relevant  information  on  process  state.  These 
need  to  cover  the  full  range  of  audible  frequencies  and  include  vibrations.  Acoustic  in¬ 
formation  from  the  process  can  be  collected  live  in  the  plant  or  generated  from  process 
measurements.  Process  sounds  can  give  a  static  acoustic^  picture  of  the  process  or  focus 
on  specific  sections  of  the  plant  according  to  operator  needs  or  current  operator  focus  and 
actions.  Individual  microphones,  placed  at  strategic  points  in  the  plant  area  can  collect  a 
live  acoustic  picture.  The  individual  signals  need  appropriate  mixing  taking  care  of  dif¬ 
ferent  sound  intensities,  thresholds  and  masking  effects.  In  contrast,  it  is  also  possible  to 
generate  a  synthetic  acoustic  picture  artificially  from  plant  data  using  stored  sound  sam¬ 
ples  or  from  noise  and  sound  generators.  In  all  cases,  the  spatial  arrangement  of  sound 
sources  must  meet  the  layout  of  the  control  room. 

As  an  example,  the  control  room  of  a  fossil-fired  power  plant  shows  a  spatial  separation 
between  furnace  and  steam  generation  and  turbine  and  power  generation.  Instruments  and 
controls  for  steam  generation  cover  the  left,  those  for  power  generation  the  right  side  of 
the  control  room.  Important  parameters  are  the  continuity  of  the  combustion  process,  the 
balance  of  heat  and  water  on  the  left  side  and  the  turbine  revolution  rate  and  vibration  on 
the  other.  In  the  past,  operators  were  able  to  feel  harmful  vibrations  of  the  turbine  in  the 
control  room  as  long  as  it  was  sufficiently  close  to  the  turbine.  The  current  load  as  well  as 
the  revolution  rate  provides  a  continuos  background  sound  in  the  turbine  building.  Mi¬ 
crophones  can  pick  up  the  sound  down  in  the  plant  and  speakers  on  the  right  side  of  the 
control  room  play  it  to  the  operators.  Special  transducers  or  low  frequency  speakers  pro¬ 
vide  vibration  information.  As  operators  have  to  keep  turbine  speed  very  close  to  the 
nominal  rate,  an  artificial  generated  signal,  providing  more  contrast  in  deviations  from  set 
points,  may  be  of  more  use.  In  this  example,  acoustic  information  provided  on  the  left 
side  of  the  control  room  relates  to  the  steam  generation  system.  As  an  example,  sounds  of 
flowing  water  with  different  intensity  can  provide  state  information.  Besides  spatial  ar¬ 
rangement,  such  sounds  show  good  contrast  to  the  noise  from  the  turbine  in  frequency 
and  complexity. 

Acoustic  information  from  the  process  control  system  includes  alarms  and  attention  get¬ 
ters,  These  can  be  generated  or  pre-recorded  tones,  sounds,  or  voice  messages.  The  sys¬ 
tem  should  provide  means  to  spatially  separate  attention-getting  audio  on  and  arrange  it 
according  the  general  control  room  layout. 

System  Components 

Following  a  market  evaluation  in  1996,  we  selected  the  equipment  and  implementation 
begins  in  late  summer  1997.  The  Hardware  selected  includes  high  performance  graphics 
workstations  as  well  as  high-end  consumer  electronics.  The  main  software  component  is 
a  set  of  programs  for  generating,  animating,  and  displaying  virtual  worlds. 

Hardware 

Central  components  of  the  hardware  are  two  graphics  workstations,  a  SGI  Octane/MXI 
with  two  195  MHz  R 10000  processors,  256  MB  memory,  4  MB  texture  memory,  and  a 
SGI  02  with  180  MHz  R5000PC  processor,  96  MB  memory,  Video  I/O  and  camera. 
Both  workstations  are  equipped  with  identical  20“  monitors  capable  of  displaying  stereo 
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and  holographic  images. 


Hardware  for  holographic  presentation  of  images  consists  of  active  stereo  glasses  with 
liquid  crystal  shutters,  an  infrared  emitter  synchronising  the  shutters  with  the  monitors 
frame  rate,  and  a  tracking  system  to  determine  the  actual  eye  point.  The  system  from 
CrystalEYES  will  be  used  in  combination  with  the  software. 

As  an  input  device,  the  Space  Mouse  will  be  used.  This  device,  developed  by  the  German 
aerospace  research  establishment  DLR,  is  basically  a  force  stick.  However,  along  with  the 
X-  and  y-axis,  it  senses  translation  force  in  the  third  dimension  (z)  and  rotational  forces  in 
three  dimensions  (pitch,  roU,  and  yaw).  A  3D  audio  system  will  be  added  in  the  future.  A 
possible  candidate  is  the  Acoustreon.  This  system,  hosted  on  a  separate  workstation,  ap¬ 
plies  signal  processing  to  given  audio  data.  Given  the  distance  and  direction  from  the  ob¬ 
server  to  the  sound  source  the  system  filters  the  audio  data  such  that  the  observer  has  the 
impression  of  a  distant  sound  source  in  this  position. 


The  conventional  audio  periphery  centres  on  a  6-channel  amplifier  and  equaliser. 


Figure  1:  System  Components 


The  two  graphics  workstations,  a  DAT  recorder,  and  microphones  provide  audio  inputs 
signals  in  stereo.  For  output,  four  speakers  are  placed  around  the  workstations  as  well  as 
headphones  for  privacy.  The  DAT  recorder  will  be  used  to  take  high-quality  samples  of 
process  noise  and  sounds  during  field  studies.  Integration  of  a  special  low-frequency  am¬ 
plifier  and  speaker  (subwoofer)  or  a  vibration  device  is  under  consideration.  Such  a  vi¬ 
bration  device,  the  T.A,N.  Vihrofloor  System  is  a  disk  with  20-cm  diameter,  coupled  to  the 
floor  or  the  operator’s  chair.  It  can  provide  vibrations  with  a  power  of  100-Watt  RMS 
with  frequencies  down  to  five  Hz. 


285 


The  video  option  of  the  02  workstation  provides  Essential  equipment  for  videoconfer¬ 
encing.  This  includes  two  input-,  one  output-channel  and  the  monitor-mounted  digital 
camera.  A  camcorder  will  be  used  for  recordings  in  plants.  We  also  imagine  the  use  of 
portable,  helmet-mounted  cameras  (for,  e.g.,  maintenance  personnel)  coupled  to  the  con¬ 
trol  room  by  radio  links.  Systems  with  a  range  suitable  for  in-house  use  are  available  at  a 
price  below  1.000  DEM.  Besides  this  local  video  equipment,  support  exists  by  the  uni¬ 
versities  Media  Centre  providing  professional  video  cutting  and  editing  facilities. 

Both  workstations  will  have  access  to  the  laboratories  local  area  network,  the  universities 
future  ATM  network,  and  the  local  ISDN  telephone  exchange.  The  laboratory’s  network 
provides  access  to  several  simulators,  among  &ose  are  a  high  fidelity  simulator  of  a  dis¬ 
tillation  column,  cement  plant  and  power  plant  simulators,  a  small  business  jet  and  a  sim¬ 
plified  helicopter. 

Software  Tools 

The  major  software  are  five  components  from  the  virtual  reality  package  of  the  German 
company  REALAX.  The  modeller  enables  to  create  3D  volumes  or  to  import  shapes  from 
conventional  CAD  data.  Objects  are  than  grouped  and  combined  to  scenes  in  the  editor, 
which  also  provides  material  properties  (colour,  reflection,  transmission)  and  surface 
textures.  The  scene  generated  this  way  is  displayed  using  the  real-time  component.  Addi¬ 
tional  software  in  combination  with  the  stereo  glasses  described  above  generates  the 
holographic  view.  Features  of  this  software  important  in  our  applications  are  level  of  de¬ 
tail  switching,  morphing,  and  callback  functions. 

Level  of  Detail  switching  saves  limited  computer  capacity.  Instances  of  an  object  are  cre¬ 
ated  with  different  complexity.  The  close-up  view  shows  the  object  with  all  its  details. 
When  the  distance  between  object  and  observer  increases,  displaying  the  less  complex 
version  keeps  the  number  of  polygons  in  the  scene  smaller. 

With  morphing,  it  is  possible  to  use  several  (slightly)  different  shapes  of  an  object  and 
display  a  weighted  average  of  these  shapes.  This  feature  is  impressively  demonstrated  in 
animated  sequence  of  car  crashes.  However,  we  can  imagine  giving  an  operator  hints  on 
pressure  and  stress  by  displaying  deformations  in  boilers,  pipes,  and  joints. 

Inevitable  in  research,  when  own  software  extends  existing  tools  or  when  simulation  data 
will  be  integrated  is  a  means  to  access  and  manipulate  of  the  visual  scene.  Callback  func¬ 
tions  hand  control  to  user  programmes  periodically  or  on  defined  events.  They  also  pass 
pointers  to  scene  data.  When  control  is  transferred  and  data  is  accessible,  the  scene  can 
manipulated  or  information  extracted  for  other  applications.  Thus,  measurements  pro¬ 
vided  from  an  external  simulator  can  change  dials  in  the  scene  and  information  on  eye 
point  position  can  be  used  for  correctly  generating  3D  audio. 

Intended  Research  Topics 

Some  of  the  features  wanted  for  a  multimedia  control  room  are  already  build  into  or  de¬ 
livered  with  the  computer  equipment,  such  as  the  InPerson  teleconferencing  tool.  This 
provides  a  shared  whiteboard,  on  which  the  local  and  different  remote  users  simultane¬ 
ously  view  and  manipulate  images  and  video  windows  displaying  the  conference  partici¬ 
pants.  However,  the  multimedia  process  control  room  is  a  research  vehicle.  Existing 
software  provide  only  tools  and  existing  solutions  from  other  areas  provide  only  pointers 
to  own  developments.  Microphones,  speakers,  and  the  camera  are  standard  equipment  of 
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modem  workstations  as  they  become  to  be  for  home  computers.  Now  it  will  be  our  task 
to  evaluate  existing  capabilities  of  hardware  and  software  as  well  as  known  solutions 
from,  for  example,  the  entertainment  industry  for  their  use  in  process  control. 

Examples  for  possible  use  of  multimedia  in  control  rooms  have  been  given  already 
above.  In  all  research  performed  in  our  laboratory  we  include  the  final  user  in  the  early 
phases.  Considering  the  growing  awareness  of  multimedia  in  present-day  Ufe  we  expect 
useful  proposals  from  future  users.  Intended  research  topics  include  multimedia- 
supported  co-operation  between  plant  operation,  maintenance,  and  management,  man¬ 
agement  of  information,  user-support  for  large  multimedia  data  bases. 

Conclusion 

The  experimental  Multimedia  Process  Control  Room  is  a  new  vehicle  to  support  future 
research  projects  of  the  Laboratory  for  Human-Machine  Systems  in  cooperation  with  in¬ 
dustrial  partners.  It  will  also  enhance  lectures  and  support  diploma-  and  PhD  theses. 
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Abstract 

In  flexible  manufacturing,  the  shop-floor  staff  is  increasingly  organized  in  semi- 
autonomous  work  groups  with  an  enriched  task  spectrum.  This  suggests  to  provide 
assistance  in  the  form  of  decision  support  systems.  However,  there  are  few  decision 
support  systems  specifically  designed  for  work  groups,  which  moreover  do  not  yet  have 
received  a  widespread  acceptance.  Therefore,  a  new  support  concept,  focusing  on  the 
universal  problem  solving  strategy  of  case-based  reasoning,  has  been  developed.  This 
concept  was  realized  as  an  application  program  for  Personal  Computers,  in  close 
cooperation  with  the  shop-floor  staff  of  a  CIM  pilot  plant.  Tlie  program  allows  to  record 
cases  in  a  format  similar  to  a  news  report,  including  multimedia  illustrations,  to  ensure  a 
vivid  representation.  The  resulting  case  database  can  be  searched  for  records  that  are 
similar  to  the  current  task,  which  then  serve  as  models.  In  addition,  the  system  provides 
functions  to  integrate  the  case  data,  to  support  the  work  group  in  its  strategic  decisions. 

Introduction 

The  increasing  complexity  of  flexible  manufacturing  systems  requires  an  integration  of 
the  human  resources,  which  takes  into  account  the  interactions  between  previously 
separated  task  domains.  Correspondingly,  there  is  a  growing  number  of  manufacturers 
who  integrate  duties  formerly  performed  by  their  specialized  departments,  like  quality 
control  and  maintenance,  into  the  task  spectrum  of  the  shop-floor  staff.  Such  an 
enrichment  of  the  task  spectrum  has  considerable  positive  effects  on  the  performance  of 
the  employees  (Lovdn,  Helander,  1997).  As  a  supplementation,  the  shop-floor  staff  is 
frequently  organized  in  semi-autonomous  work  groups,  coordinating  the  activities  and 
pursuing  a  continuous  improvement  process.  As  an  example,  in  the  German  machine 
building  industry  the  proportion  of  companies  practicing  group  work  in  their 
manufacturing  departments  increased  from  29%  to  43%  between  1991  and  1993  (SFB 
187,  1994,  p.  3).  That  is,  substantial  expert  decision  competencies  are  transferred  to  the 
shop-floor. 

Due  to  the  complexity  of  flexible  manufacturing  systems,  it  seems  to  be  advisable  to 
provide  the  shop-floor  staff  with  appropriate  decision  support  tools,  as  an  assistance  in 
exercising  its  new  competencies.  However,  most  of  the  currently  available  decision 
support  systems  are  primarily  designed  to  fit  the  requirements  of  specialized 
departments.  Presupposing  a  sophisticated  educational  background,  these  tools  represent 
the  work  system  in  an  abstract,  generalized  way.  In  addition,  they  are  rather  incompatible 
with  an  integrated  task  spectrum,  because  of  their  frequently  narrow  field  of  application. 

Lately,  a  number  of  decision  support  systems  specifically  designed  for  a  shop-floor 
environment  have  been  introduced,  to  assist  work  groups  in  short-term  scheduling 
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(Mertins,  Carbon,  1996),  quality  management  (Triebe,  Falter,  Krings,  1996;  Wasserman, 
1995),  or  failure  diagnosis  and  repair  (Engel,  1996;  Timpe,  Rothe,  GaBner,  1997).  But 
there  are  indications  that  these  systems  meet  the  practical  requirements  on  the  shop-floor 
only  to  a  certain  extent.  As  a  current  example,  in  the  research  project  RE-INST, 
concerned  with  'Job  Enrichment  by  Reintegration  of  Maintenance  Tasks ,  the 
participating  companies  had  to  resort  to  internal  developments,  which  were  more  or  less 
ad-hoc  solutions,  because  of  the  limited  software  development  resources  (lAW,  1997). 

Therefore,  in  the  following  section  weak  points  of  the  currently  available  decision 
support  systems  and  potential  alternative  design  concepts  will  be  discussed.  Subsequent 
to  this  discussion  a  prototypical  realization  of  the  proposed  design  alternatives  is 
outlined.  The  final  section  addresses  possible  shortcomings  of  this  prototype. 

Current  Systems  and  Alternative  Design  Concepts 

Currently  available  decision  support  systems  for  work  groups  are  frequently 
characterized  by  three  weak  points,  a  restricted  scope  of  application,  an  insufficient 
flexibility,  and  a  tutorial  support  concept.  Generally,  the  systems  are  designed  to  assist  a 
single  task,  for  example  maintenance,  whereas  the  enriched  task  spectrum  of  the  shop- 
floor  staff  requires  a  more  comprehensive  support.  Moreover,  they  are  frequently 
restricted  to  specific  equipment,  for  example  a  certain  manufacturing  center,  which 
seriously  limits  the  set  of  potential  users  and  provides  difficulties  for  the  integration  of 
the  system  in  the  information  infrastructure  of  the  factory. 

A  solution  to  the  problem  of  a  restricted  scope  demonstrates  SAP,  for  example  in  its 
latest  product  R/3.  Here,  the  software  is  a  rather  general  frame  that  the  users  tailor  to 
their  needs  and  requirements  by  entering  the  equipment  and  company  data  relevant  for 
their  purposes. 

However,  experts  are  necessary  to  adapt  such  an  installation  to  modifications  of  the 
equipment  or  the  organization.  This  limited  flexibility  carries  a  lot  of  weight  in  the 
present  context,  as  work  groups,  in  their  continuous  improvement  process,  are  expected 
to  identify  weak  points  of  their  work  system  and  to  eliminate  them  rather  autonomously, 
whenever  possible.  Therefore,  it  would  be  more  coherent  if  the  shop-floor  staff  could 
represent  these  improvements  in  the  corresponding  databases  of  their  assistance  system. 

Finally,  the  support  concept  of  currently  available  assistance  software  is  generally  based 
on  the  expert  system  approach.  That  is,  the  knowledge  of  external  experts  about  the 
equipment  and  the  task  domains  is  implemented  in  databases,  which  the  shop-floor  staff 
uses  as  more  or  less  intelligent  computerized  manuals.  However,  such  a  tutorial  support 
concept  is  rather  incompatible  with  the  motivation  to  introduce  group  work,  to  tap  the 
staffs  practical  knowledge  and  make  this  expertise  available  to  the  colleagues. 

An  alternative  support  concept  can  be  derived  from  the  finding  that  expert  employees 
frequently  accomplish  a  task  in  analogy  to  a  previous,  similar  case  (Rothe,  Timpe,  1997). 
According  to  Riesbeck  and  Schank  (1989)  this  principle  is  a  ubiquitous  problem  solving 
strategy,  or  as  they  say,  'the  essence  how  human  reasoning  works'  (p.  7). 

The  concept  of  case-based  reasoning  is  outlined  in  Figure  1 .  Supposing  that  an  employee 
realizes  that  the  fill-level  of  an  oil  gauge  is  only  75%  (if-component  of  the  current  case). 
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Then  the  question  arises  how  he  or  she  should  react  to  this  finding  (then-component  of 
the  current  case).  To  clarify  this  point,  one  can  refer  to  episodic  memory,  in  which 
numerous  previous  cases  are  stored,  each  as  an  if-then-relation.  First  he  or  she  tries  to 
recall  the  case  whose  if-component  approximates  to  the  if-component  of  the  current  case 
(Best  Match).  Subsequently,  the  employee  has  to  adapt  the  then-component  of  the 
recalled  case  to  the  one  under  consideration  (Analogy  Formation).  In  the  scenario  in 
Figure  1 ,  for  example,  he  or  she  would  refill  a  smaller  amount  of  oil,  as  the  fill-level  is 
now  a  little  bit  higher  than  that  in  the  recalled  case. 


Figure  1:  Conceptual  outline  of  case-based  reasoning 


According  to  the  outline  in  Figure  1  an  assistance  system  supporting  case-based 
reasoning  should  have  at  least  three  components.  First,  the  shop-floor  staff  needs  a 
database  containing  previous  cases,  as  a  reduction  of  the  memory  demands.  Secondly, 
the  system  should  provide  a  function  allowing  to  retrieve  a  case  whose  if-component  fits 
the  current  task,  as  a  support  of  the  best  match.  Thirdly,  the  system  ought  to  assist  in  the 
adaptation  of  the  then-component  of  the  retrieved  case  to  the  current  conditions,  the 
analogy  formation. 

As  the  case  database  represents  the  practical  knowledge  of  the  shop-floor  staff  it  cannot 
be  provided  by  external  experts.  Rather,  the  shop-floor  staff  should  record  the  cases  they 
accomplished,  which  requires  an  equipment-  and  task-independent  data  recording 
interface.  In  addition,  the  cases  are  to  be  represented  as  illustrative  as  possible,  to  allow 
that  they  can  be  reconstructed  by  those  colleagues  who  were  not  directly  involved,  as 
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well  (Proctor,  Dutta,  1995).  This  requirement  of  a  vivid  representation  leads  to  the 
application  of  multimedia  (Hasebrook,  1995).  That  is,  cases  should  be  illustrated  by 
graphic,  sound,  or  digital  video  files,  if  necessary. 

However,  the  work  group  needs  not  only  support  in  the  execution  of  specific  tasks,  but 
also  in  its  strategic  decisions  concerning,  for  example,  quality  control  procedures  or 
preventive  maintenance  intervals.  With  regard  to  the  requirements  of  such  a  support, 
Lockamy  and  Cox  (1995)  claim  that  ’at  the  plant  level,  performance  on  the  strategic 
objectives  must  be  evaluated  with  a  minimal  passage  of  time  to  provide  an  appropriate 
level  of  control’  (p.  233).  Thus,  the  decision  support  should  also  supply  functions  for  the 
integration  of  the  case  data,  allowing  to  perform  weak  point  analyses. 

Prototypical  Realization 

The  alternative  design  concepts  outlined  above  have  been  realized  in  a  Windows 
application  program  for  Personal  Computers.  The  program,  called  Behavior-Outcome 
Feedback  (BOF),  has  been  developed  in  a  CIM  pilot  plant,  in  close  cooperation  with  the 
shop-floor  staff,  which  in  addition  tested  different  versions  in  practice.  BOF  is  installed 
directly  at  the  equipment  and  thus  is  integrated  in  the  activities  on  the  shop-floor.  The 
program  provides  a  facility  for  the  recording  and  administration  of  cases.  These  cases  are 
merged  by  feedback  functions,  presenting  the  underlying  behavior-outcome 
contingencies  in  a  graphic  way.  In  the  following,  the  use  of  the  program  is  illustrated  by 
referring  to  data  that  were  recorded  at  the  flexible  manufacturing  center  Huller-Hille  nb-h 
90, 

The  desktop  is  designed  as  a  Multiple  Document  Interface  (MDI),  the  case 
administration  and  feedback  charts  are  child  windows  within  the  main  program  window. 
Figure  2  shows  the  child  window  of  the  case  administration.  Its  structure  is  based  on  text 
grammars  for  news  reports  (Dijk,  1983),  to  provide  a  format  that  is  general  enough  to 
document  cases  from  different  task  domains.  The  field  Condition  describes  the  state  of 
the  object  under  consideration  before  and  after  the  event  or  activity.  These  values  do  not 
necessarily  differ,  in  an  inspection,  for  example,  they  are  identical.  Field  Cost  allows  to 
record  aspects  relevant  for  the  evaluation  of  the  event  or  activity,  which  typically  are 
economic  variables.  Beyond  that,  they  may  include  features  like  the  cognitive  or  physical 
strain,  if  one  advocates  a  more  general  cost  concept  (Fiy,  1995;  Gelders,  Manaerts,  Maes, 
1994). 

The  values  for  the  fields  Status,  Who,  What,  Where,  Why,  Condition,  and  Cost  are 
entered  using  dialogs  in  which  the  alternatives  are  organized  in  hierarchical  selection 
lists,  similar  to  the  Explorer  in  Windows  95  and  NT.  These  selection  lists  can  be  edited, 
permitting  to  adapt  the  case  administration  to  arbitraiy  equipment,  task  spectra,  to  be 
controlled  conditions,  and  cost  aspects.  For  example,  one  might  record  not  only 
maintenance  activities,  as  shown  in  Figure  2,  but  also  quality  control  measures, 
specifying  the  corresponding  items. 

The  section  Illustration  allows  to  attach  multimedia  files  depicting  a  specific  aspect  of 
the  case  in  a  more  vivid  way.  The  program  processes  a  fairly  wide  spectrum  of 
multimedia  data,  be  it  audio,  bitmap,  or  digital  video  files.  Such  a  file  might  be  a 
scanned  technical  drawing  showing  the  construction  of  a  component  or  a  tutorial  video 
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about  how  to  perform  a  specific  measure.  Pushing  a  button  in  the  Illustration  section,  the 
multimedia  file  is  displayed  in  its  original  size  in  a  separate  child  window,  which 
provides  some  extra  presentation  functions. 


^"'{Behavioi-Outcome  Feedback 


file  £dil  £ases  ^election  Inlegralion  Mu^^imedia  jgindow  Help 


HS^ 


_ 

1  1^;  Cases  -  CABOF\EXAMPLE\NBH90.DBF 

ime  I  Roirtine  cleaning  of  our  'home-made' fitter 
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1 

. . . 
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Creates  a  new  table  with  cases. 


Figure  2:  Main  window  of  EOF  with  the  child  window  for  the  administration  of  cases, 
presenting  a  maintenance  activity 


Case-based  reasoning  is  supported  particularly  by  a  table  function.  In  such  a  table  the 
previous  cases  relevant  for  accomplishing  the  current  task  are  listed.  Figure  3  shows  a 
table  for  the  question  with  which  soiling  the  filter  of  the  chip  scavenging  system  has 
been  cleaned  so  far.  In  addition,  one  of  these  cases  provides  an  Illustration  When  to 
clean’,  showing  the  cleaning  criterion  in  a  digital  photo.  For  an  easy  access  of  the 
complete  case  data,  tables  provide  a  navigation  facility  -  clicking  on  an  item  the 
corresponding  record  is  presented  in  the  child  window  of  the  case  administration. 

Other  functions  perform  an  integration  of  the  cases,  to  support  strategic  decisions  of  the 
work  group.  One  aspect  is  to  aggregate  the  cost  values,  in  order  to  determine  weak  points 
of  the  strategies.  An  example  of  a  cost  analysis  is  shown  in  Figure  4,  presenting  the 
working  time  expenditures  for  the  maintenance  of  the  coolant  system  components  during 
a  specified  period  of  time.  According  to  this  bar  chart,  exceptional  working  time  values 
were  recorded  for  maintaining  the  tank,  as  compared  to  the  other  components  of  the 
machine  unit. 
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Again,  BOF  is  rather  flexible,  as  it  allows  to  analyze  arbitrary  subsets  of  the  selection 
lists.  Moreover,  the  case  data  can  be  analyzed  with  regard  to  different  levels  of  the  cost 
hierarchy.  For  example,  the  program  permits  to  specify  a  weighted  sum  of  costs,  rather 
than  a  single  item,  as  the  outcome  variable. 


When  to  clean?  -  CAB0F\EXAMPLEANBH90.DBF 


When  to  clean? 
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Component/Soiling  [%]... /20  20 

Maintain/Ciean 

Maintain/Inspect 

Maintain/Inspect 

Maintain/Clean 

Maintain/Inspect 

Maintain/Clean  1 

Maintain/Inspect 

Maintain/Inspect 

Illustration 


Instruction  as  text 


When  to  clean 


Instruction  a 


Figure  3:  Table  of  previous,  similar  cases  for  the  problem  when  to  clean  the  filter  of 
the  chip  scavenging  system;  the  case  in  the  last  row  represents  the  corresponding 
current  case 
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Figure  4:  Cost  analysis  presenting  working  times  for  maintenance  activities  at  the 
components  of  the  coolant  system  during  a  specified  period  of  time 


If  a  weak  point  has  been  determined,  the  next  step  is  to  identify  potential  reasons.  This  is 
supported  by  providing  time  and  condition  analyses.  A  time  analysis  allows  to  determine 
whether  an  event  or  activity  occurred  periodically,  after  a  constant  time  interval.  For  this 
purpose,  the  interval  of  successive  cases  is  plotted  as  a  function  of  time,  expecting  that 
they  form  a  more  or  less  parallel  line  to  the  time  axis.  Figure  5  shows  the  corresponding 
scatter  plot  for  the  events  at  the  coolant  system  tank,  which  are  obviously  aperiodic.  As 
in  the  tables,  such  a  time  analysis  includes  a  navigation  facility;  pushing  a  circle  one  gets 
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the  corresponding  complete  record  in  the  child  window  of  the  case  administration.  This 
would  reveal  that  the  events  in  Figure  5  were  concerned  with  the  inspection  of  the  fill- 
level  and  the  topping  up  of  the  tank.  Therefore,  a  potential  cause  of  the  identified  weak 
point  is  an  inconsistently  performed  time-based  strategy. 


Time  analysis  of  lank  mainlenance  -  C:\BOF'vEXAMPLE\NBH90.DBF  gElB 


Time  ana^is  of  tank  maintenance 

Cost  =  Sum  [$]/Working  time  [min].../,  When  =  01 .07.94  00:00:00  -  30.06.95  00:00:00,  Editor  =  BOF/GUDE/,  State  =  26.08.97  ... 
■  Parallel  Cases:  None 

#  Intervall  Cases:  VMiere  =  nb-h  90/Coolant  system/Tank 


Figure  5:  Time  analysis  for  the  maintenance  of  the  coolant  system  tank  during  a 
specified  period  of  time 


However,  an  alternative  explanation  is  that  the  maintenance  of  the  tank  was  not 
performed  as  a  routine  task,  but  rather  dependent  on  its  condition.  The  shop-floor  staff  of 
the  CIM  pilot  plant  preferred  such  condition-based  strategies,  as  can  be  seen  from  an 
analysis  of  their  recorded  maintenance  activities.  According  to  this,  52%  of  the  cases 
were  inspections.  Moreover,  condition-based  service  and  condition-based  preventive 
repair  dominated  over  their  time-based  variants  (26%  vs.  6%  and  2%  vs.  1%, 
respectively).  Finally,  11%  of  the  records  were  corrective  repairs.  The  preference  of 
condition-based  strategies  seems  to  be  rather  general  in  nature,  as  the  above  finding  is  in 
accord  with  results  from  more  controlled  laboratory  settings  (Gude,  Psaralidis,  Stiegler, 
Seeber,  1996;  Kerstholt,  1995). 

Consequently,  the  assistance  system  permits  to  create  so-called  condition  analyses.  Such 
an  analysis  visualizes  whether  an  event  or  activity  occurred  with  specific  values  of  the 
condition  under  consideration.  Thus,  for  a  consistently  performed  condition-based 
strategy  one  would  expect  that  the  cases  line  up  on  a  parallel  to  the  time  axis. 

Figure  6  shows  the  corresponding  plot  for  the  fill-level  of  the  coolant  system  tank.  A 
circle  represents  the  condition  before  an  activity,  the  top  of  the  arrow  depicts  the 
resulting  condition.  According  to  the  figure,  during  the  first  six  months  there  was  no  well 
defined  criterion  when  to  top  up  the  tank,  occasionally  this  activity  was  performed  rather 
late.  During  the  last  six  months  the  behavior  was  much  more  consistent,  with  the 
criterion  beyond  a  fill-level  of  50%. 
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This  pattern  suggests  that  the  identified  weak  point  is  based  on  an  initial  inconsistent 
condition-based  maintenance.  To  test  this  hypothesis,  the  staff  can  create  separate  cost 
analyses  for  the  first  and  last  six  months.  These  analyses  reveal  that  during  the  first 
period  of  time  the  working  time  expenditure  was  1 82  min,  which  then  decreased  to  87 
min.  That  is,  the  strategy  shift  in  1995  had  actually  a  beneficial  effect  on  the  outcome 
variable  analyzed  here. 


Condiliun  analysis  ol  lank  maintenance  -  C:SB0F\EXAMPLE\NBH90.DBF  _ 


Condition  analysis  of  tank  maintenance 
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■  Parallel  Cases:  None 
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Figure  6:  Condition  analysis  for  the  maintenance  of  the  coolant  system  tank  during  a 
specified  period  of  time;  the  three  cases  in  Figure  5  for  the  period  of  time  between 
23.11.94  and  04.02.95  are  not  displayed  here,  as  the  required  filldevel  data  are  missing 
in  these  records 


01.07.94  11.09.94  23.11.94  04.02.95  18.04.95  30.06.95 

When 


Condition  analyses  do  not  only  assist  in  the  evaluation  of  preventive  maintenance 
strategies  but  can  be  applied  in  quality  control  as  well.  The  corresponding  plot  is  similar 
to  a  Shewhart  control  chart  in  Statistical  Process  Control  (SPC),  indicating  required 
adjustments  or  replacements  of  tools.  Moreover,  examining  the  quality  control 
procedures  in  cost  analyses  permits  to  optimize  the  sampling  frequency  as  well  as  the 
sample  size. 

Conclusions 

As  a  solution  to  the  problems  of  a  restricted  scope  of  application  and  an  insufficient 
flexibility,  EOF  provides  a  framework  that  the  work  groups  can  adapt  to  their  specific 
requirements.  As  an  alternative  to  the  commonly  realized  tutorial  support  concept,  the 
shop-floor  staff  is  asked  to  record  its  practical  knowledge  in  a  case  database,  which 
supports  the  universal  problem  solving  strategy  of  case-based  reasoning.  The  database 
reduces,  inter  alia,  the  memory  demands,  which  have  been  identified  as  a  critical 
performance  variable  in  manufacturing  (Stanislaw,  1995).  Moreover,  less  experienced 
employees  can  utilize  their  colleagues*  practical  knowledge,  especially  if  the  cases  are 
illustrated  by  multimedia  files.  Finally,  the  program  allows  to  integrate  the  case  data  for 
weak  point  analyses,  which  support  the  strategic  decisions  during  the  continuous 
improvement  process. 
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The  multimedia  files  are  not  provided  by  the  development  team,  as  the  specific  field  of 
application  is  not  known  beforehand.  Rather,  the  users  have  to  get  or  produce  those  files 
that  are  relevant  for  their  purposes.  However,  taking  into  account  the  current  technical 
state-of-the-art,  one  should  not  expect  that  complex  multimedia  presentations  can  be 
produced  by  the  work  groups,  because  specific  expert  knowledge  is  still  required.  On  the 
contrary,  this  is  a  task  for  specialized  departments  or  the  manufacturer  of  the  equipment. 
Nevertheless,  with  the  increasing  application  of  multimedia  in  the  office  and  at  home,  it 
is  very  likely  that  the  required  knowledge  will  be  generally  available  in  the  near  future. 

In  the  current  version  of  the  assistance  system  the  execution  of  the  best  match  and  the 
analogy  formation  is  left  to  the  shop-floor  staff.  The  program  allows  to  flexibly  create  a 
selection  of  previous  cases.  However,  the  user  has  to  decide  which  of  these  cases 
optimally  matches  the  present  task  and  how  to  adapt  it  to  the  current  conditions.  For  the 
moment,  an  automation  of  these  processes  has  been  deferred,  as  it  presupposes 
considerable  knowledge  about  the  specific  field  of  application  (Reimann,  1997). 

Although  data  recording  was  held  at  a  minimum,  it  still  requires  some  effort.  Therefore, 
the  users  should  try  to  achieve  an  additional  return  on  investment.  In  particular,  the 
exchange  of  case  data  ought  not  to  be  restricted  to  the  members  of  a  single  work  group 
but  rather  is  to  be  extended  across  groups  and  companies,  including  the  manufacturers  of 
the  equipment.  Such  an  integration  would  be  especially  valuable  for  small  and  medium- 
size  companies,  which  frequently  have  only  restricted  opportunities  to  obtain  the 
practical  knowledge  required  to  develop  optimized  manufacturing  strategies. 
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Abstract 

Today  the  consideration  of  worker,  costs  and  environmental  factors  becomes  more  and 
more  important  in  designing  of  technical  systems.  Diverse  criteria,  like  qualification, 
work  organisation,  compensation  of  employees  and  work  structuring,  need  to  be  more 
integrated  into  the  planning  process  and  for  that  reason  also  into  the  planning  tools. 

The  qualification  of  workers  is  very  important  for  reengineering  manufacturing  systems. 
Material  flow  simulators  generate  often  only  results  concerning  the  utilization  of  work¬ 
ers.  The  developed  evaluation  module  takes  into  consideration  to  what  extent  a  worker  is 
qualified  for  a  given  activity.  It  compares  the  qualification  of  the  worker  with  the  re¬ 
quirements  of  the  working  process.  The  interpretation  of  the  results  allows  the  determi¬ 
nation  of  necessary  training  for  the  workers  already  in  the  planning  stage. 

The  project  is  supported  by  the  BMBF  (German  Federal  Ministry  of  Education,  Science, 
Research  and  Technology)  in  its  program  „Work  and  Technology*'. 

Introduction 

In  the  last  years  many  companies  underwent  a  change  from  traditional  structures,  like 
productions  in  bulk,  toward  customers'  order  production.  This  requires  new  concepts  of 
production  organisation  and  flexible  working  structures.  Planning  and  simulation  tools 
must  be  able  to  describe  these  structures.  In  these  kind  of  systems  and  their  models, 
workers  become  more  important. 

Classic  simulators  are  able  to  model  technical  processes  in  manufacturing  and  material 
handling  systems  with  a  variable  degree  of  detail.  They  are  used  by  factory  planners  to 
check  and  improve  plans  of  technical  systems.  During  the  last  years  these  tools  were  ex¬ 
tended  with  components  to  support  the  modelling  of  flexible  manufacturing  systems, 
automated  production  lines  and  automated  material  flow  processes. 

One  of  the  main  application  areas  of  factory  simulation  is  the  modelling  of  technical 
systems  with  their  production  processes  and  material  flow.  But  today  the  consideration  of 
worker,  costs  and  environmental  factors  becomes  more  and  more  important  in  the  design 
of  technical  systems.  Diverse  criteria,  like  qualification,  work  organisation,  compensa¬ 
tion  of  employees  and  work  structuring,  need  to  be  more  integrated  into  the  planning 
process  and  for  that  reason  also  into  the  planning  tools. 
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Background 

Today’s  simulation  tools  are  equipped  with  extensive  components  for  the  assessment  of 
the  behavior  of  manufacturing  and  material  flow  systems.  The  simulation  results  are  de¬ 
livered  in  a  processed  form  as  compressed  result  data.  They  include,  for  instance,  infor¬ 
mation  on  the  utilization  rate  of  transport  devices  and  manufacturing  facilities,  the  utili¬ 
sation  of  storage  space,  or  the  throughput  time  of  items  or  orders.  These  data  allow  the 
materials  flow  planner  an  effective  assessment  of  technical  systems,  their  performance 
and  their  performance  limits.  Besides,  most  simulators  offer  primary  result  data.  These 
are  for  instance  trace  protocols.  The  animation  trace  file  is  a  special  form  of  primary  re¬ 
sult  data  which  allows  a  post-run  animation.  With  the  help  of  the  animation  components 
in  today’s  simulation  tools,  the  experienced  user  makes  use  of  various  possibilities  in 
order  to  assess  the  modelled  processes  and/or  to  put  the  system  states  in  a  proper  place 
due  to  their  history. 

Different  levels  of  description  and  modelling  for  the  workers  are  possible.  In  traditional 
simulation  models  workers  are  often  only  components  required  for  the  operation  of  the 
manufacturing  system.  This  kind  of  description  now  is  not  sufficient  to  fulfil  the  ex¬ 
tended  requirements  of  simulation  studies. 

A  team  of  the  Association  of  German  Engineers  (German  abbreviation:  VDI)  is  devel¬ 
oping  a  VDI  Guideline  dealing  with  the  „Simulation  of  material  flow  and  production 
systems“.  It  is  known  as  VDI  3633.  Part  8  of  this  Guideline  deals  with  the  workers  in 
simulation  models.  Two  terms  are  introduced:  worker-integrated  and  worker-oriented 
simulation.  The  main  aim  of  this  part  is  to  define  the  minimum  requirements  for  worker- 
integrated  as  well  as  worker-oriented  simulation  of  manufacturing  and  material  handhng 
systems. 

Worker-integrated  simulation  requires  in  addition  to  the  modelling  of  the  production  and 
material  handling  system  also  the  modelling  of  the  workers. 

For  the  modelling  of  the  workers  the  following  minimum  requirements  have  been  de¬ 
fined: 

•  description  of  each  single  worker  or  type  of  worker, 

•  description  of  teamwork, 

•  consideration  of  the  qualification  level  of  each  worker  or  type  of  worker  and 

•  separation  between  running  time  of  technical  system  and  working  hours  of  the  work¬ 
ers. 

hi  addition  to  the  classic  simulation  results,  the  planning  team  is  provided  with  more 
information  about  the  system  for  their  decision  making,  e.g.  they  can  estimate  how  many 
workers  are  necessary  and  what  qualification  they  should  have.  This  kind  of  modelling  of 
workers  only  requires  an  appropriate  detailed  modelling  of  the  technical  system. 

We  use  the  term  worker-oriented  simulation  in  case  that  the  simulation  study  primarily 
deals  with  worker  related  aims.  Again,  the  basis  is  the  model  of  the  production  and  mate¬ 
rial  handling  system.  Normally,  the  level  of  detail  is  higher  than  for  worker-integrated 
simulation. 
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Aims  of  the  simulation  study 
are  to  get  results  about  impacts 
of  different  working  forms  on 
workers.  Therefore,  the  model¬ 
ling  of  additional  properties  is 
necessary.  These  are,  for  exam¬ 
ple, 

•  physical  and  non-physical 
stress  and  strain  factors  for 
workers, 

•  changes  in  qualification  due 
to  learning  and  unlearning, 
or 

•  modelling  of  worker's  reli- 
abihty. 

The  modelling  of  these  proper¬ 
ties  requires  additional  input 
data  and  evaluation  algorithms. 

Additional  input  data  and 
evaluation  components  are  ex¬ 
pensive  and  so  only  properties 
relevant  for  the  aims  of  the  1:  Data  Flow  in  Advanced  Simulation 

simulation  study  are  included.  Models 

In  both  cases,  the  worker-integrated  and  the  worker-oriented  simulation,  the  results  about 
the  workers  have  to  be  seen  in  combination  with  the  results  about  the  technical  systems. 

Extension  of  Simulation  Tools 

The  Otto-von-Guericke  University  of  Magdeburg  and  the  Institute  for  Industrial  Medi¬ 
cine,  Safety  and  Ergonomics  (ASER)  of  Wuppertal  have  been  cooperating  in  the  field  of 
worker- oriented  simulation  for  about  6  years  now.  Main  aspects  are  the  modelling  of 
stress  and  strain  as  well  as  of  qualification  aspects.  We  prototypically  develop  and  test 
modules  to  complement  simulation  tools  with  defined  interfaces.  As  these  modules  are 
useful  for  a  wide  area  of  simulation  languages  and  simulation  packages,  a  high  user  ac¬ 
ceptance  can  be  achieved. 

The  precondition  for  getting  additional  information  is  a  simulation  model  that  describes 
the  analysed  system  in  a  very  detailed  way.  It  must  have  a  clear  connection  of  all  manu¬ 
facturing  sequences  to  machines,  items  and  workers.  The  series  of  manufacturing  se¬ 
quences  (processes)  are  to  be  recorded  in  trace  protocols.  Also  the  writing  of  trace  proto¬ 
cols  of  changes  in  states  of  system  components  must  be  possible.  Both  forms  of  trace 
protocols  constitute  as  primary  result  data  the  basis  for  the  extended  data  evaluation. 

In  principle,  two  approaches  are  possible:  Additional  functions  may  be  integrated  in 
simulation  tools,  as  is  done,  for  example,  in  simulators  with  special  economic  evaluation 
components.  Such  an  approach  requires  a  special  tool  for  each  simulator. 
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A  second  approach  is  based  on  post-run  data  interpretation.  The  unprocessed  trace  data 
are  connected  with  additional  input  data.  Special  modules  allow  the  extended  evaluation 
of  the  system  (see  Figure  1),  Different  views  for  different  user  groups  are  possible. 

We  used  the  second  approach.  As  an  environment  for  the  simulator,  the  evaluation  mod¬ 
ules  and  the  database  we  use  a  help  and  documentation  system.  The  following  basic 
functions  are  part  of  the  system: 

•  collection  and  interpretation  of  input  data 

•  evaluation  of  the  results 

•  help  and  documentation 

•  database  connection 

The  evaluation  modules  can  be  chosen  according  to  the  aims  of  the  simulation  study. 
Figure  2  shows  the  implemented  modules  and  their  interfaces. 

Help  and  documentation  system  as  an  environment  for  simulation 
Normally  the  users  of  simulation-models  are  engineers  of  planning  divisions,  e.g,  mate¬ 
rial  flow  engineers.  This  group  of  users  is  often  not  so  familiar  either  with  the  developing 
of  a  model  or  with  the  interpretation  of  ergonomic  results.  Therefore  it  is  necessary  to 
give  assistance  in  all  states  of  the  simulation  process. 

The  prototypicaUy  developed  help  and  documentation  system  was  designed  in  Tool- 
book™  as  an  environment  for  supporting  a  planning  team  during  the  process  of  changes 
in  working  structures.  The  aim  is  not  to  replace  the  planning  team,  but  to  give  them  ad¬ 
vice  in  decision-making.  Additional  specific  data  and  information  can  be  integrated  by 
the  user. 

The  help  and  documentation  system  may  be  used  in  each  phase  of  restructuring.  In  the 
first  stage  experiences  from  similar  projects  in  other  companies  are  demonstrated  so  that 
the  planning  team  can  put  their  aims  in  concrete  terms.  They  are  then  asked  for  necessary 
input  data  to  give  a  rough  estimate  of  the  planned  system. 

The  system  demonstrates  to  the  users  the  possibilities  and  the  limits  of  a  simulation  study 
for  a  special  field  of  application.  Based  on  this  information,  the  planning  team  can  de¬ 
cide  whether  a  simulation  study  is  necessary  or  not. 

If  more  detailed  information  of  the  dynamic  process  is  necessai^,  input  data  that  have 
been  introduced  until  this  stage  can  be  transferred  into  the  simulation  model.  Examples 
for  such  data  are  the  description  of  the  working  cycles  and  their  characteristics,  types  of 
workers  and  their  qualification  as  well  as  specifications  of  the  expected  order  situation. 

After  the  simulation  runs,  the  help  and  documentation  system  supports  the  interpretation 
of  the  results.  Another  feature  of  the  system  is  to  document  the  restructuring  process. 
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Figure  2:  Components  and  Interfaces  of  the  System 


Collection,  preparation  and  management  of  input  data 

In  addition  to  the  data  collection  supported  by  the  help  and  documentation  system,  spe¬ 
cial  modules  can  be  used.  Evaluated  data  for  stress  factors  of  working  sequences  can  be 
directly  imported  from  the  stress  documentation  system  (German  abbreviation:  BDS) 
which  was  developed  for  risk  assessment  in  enterprises.  Also  special  input  data  modules 
as  a  part  of  simulators  can  be  used. 

The  data  are  transferred  to  a  common  database,  in  our  case  we  use  the  PARADOX™- 
format.  For  the  data  exchange  between  database  and  simulator  we  use  the  ASCII-Format. 
This  format  is  supported  by  nearly  every  database  and  simulation  tool  independent  from 
the  hardware  and  operation  system. 

For  the  extended  evaluation  of  the  simulation  results,  it  is  necessary  to  find  a  reference 
parameter  the  additional  properties  can  be  assigned  to. 

The  application  of  this  concept  requires  a  trace  file  generated  by  the  simulator  with  the 
contents  starting  and  closing  time,  working  location,  working  item  and  number  of  worker 
for  each  working  sequence.  The  combination  of  working  sequences  and  primary  simula¬ 
tion  results  is  realized  by  a  connection  file  which  is  represented  by  a  matrix. 

Another  kind  of  data  connection  is  the  extension  of  state  protocols  generated  by  the 
simulator  with  additional  data.  These  state  protocols  are  necessary  for  the  evaluation  of 
data  which  cannot  be  collected  from  working  sequences,  e.g.  environmental  conditions. 

One  of  the  main  aspects  of  our  work  is  to  get  more  information  on  the  required  qualifi¬ 
cation  of  the  workers.  Questions  that  arise  often  during  the  restructuring  process  are: 

•  Are  the  workers  qualified  for  the  new  tasks  ? 
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•  Which  tasks  must  a  worker  be  qualified  for? 

It  is  difficult  to  generate  results  about  the  required  qualification  of  the  workers.  Normally 
the  simulator  assigns  a  worker  to  a  work  task  only  if  he  has  an  adequate  qualification. 
However,  simulators  do  not  consider  the  effects  of  qualification  on  task  duration  and 
product  quality.  If  no  worker  with  the  required  qualification  is  available,  the  process  is 
suspended  until  such  a  worker  is  idle. 


Figure  3:  Input  data  collection  using  a  help  and  documentation  system 
here:  Input  of  worker's  qualification  level 

In  our  modules  two  approaches  are  realized:  The  first  approach  is  to  give  each  worker  a 
very  high  qualification.  During  the  post-run  evaluation,  we  analyze  to  what  extent  each 
single  qualification  was  used.  Specialists  interpret  the  results  and  determine  the  neces¬ 
sary  qualification  demands  for  the  workers. 

In  the  second  approach,  the  model  control  of  the  simulator  gets  the  qualification  data  by 
a  number  for  each  worker  and  each  task  in  the  range  from  „0“  (not  able  to  perform  the 
task)  to  „1“  (well  qualified  for  this  task)  (see  Figure  3).  Different  levels  of  qualification 
can  be  used  for  the  assignment  of  workers.  If  the  chosen  worker  is  not  so  well  qualified, 
this  may  influence  the  duration  of  performance  in  the  model-  The  interpretation  of  the 
results  allows  the  determination  of  necessary  training  for  the  workers  already  in  the 
planing  stage. 

Integration  of  physiological  aspects 

In  particular  in  case  of  heavy  work  load  or  aggravated  environmental  conditions  physio¬ 
logical  aspects  should  be  incorporated  into  the  simulation  model  to  provide  for  the 
evaluation  of  the  workers’  stress  and  strain  factors  already  in  the  planning  phase  and  the 
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prevention  of  overstrain. 


Therefore  different  stress  factors  must  be  taken  into  consideration.  They  result  from  the 
task,  on  the  one  hand,  and  the  environmental  conditions,  on  the  other.  Stress  factors 
caused  by  the  task  are,  for  example,  dynamic  and  static  muscular  workload  as  well  as  the 
required  body  posture.  Stress  factors  caused  by  environmental  conditions  are,  for  exam¬ 
ple,  noise,  vibrations,  heat  or  cold.  They  may  primarily  depend  on  the  working  location, 
but  also  on  the  working  item  and  the  task  itself. 

The  level  of  detail  should  allow  the  definite  assignment  of  working  sequences  to  any 
workers,  together  with  the  indication  of  duration  and  working  location.  A  working  se¬ 
quence  may  be  composed  of  several  activities,  but  may  not  be  interrupted  in  the  simula¬ 
tion  model.  The  required  data  may  be  derived  from  risk  assessment  studies,  which  has  to 
be  done  due  to  European  regulations.  If  no  data  are  available,  instruments  for  risk  as¬ 
sessment  studies  (e.g.  BDS)  can  be  used  in  a  prospective  way. 

As  a  result  of  a  simulation  study  we  get  exposure  times  for  each  worker.  By  combining 
the  stress  intensity  and  the  exposure  times,  we  can  evaluate  the  stress  factors  for  a  normal 
shift.  Physiological  models  for  the  prediction  of  heart  rate  are  available  to  take  synergetic 
effects  into  account. 

By  incorporating  such  ergonomic  considerations,  the  planning  team  gains  information  on 
potential  risks  in  an  early  stage  of  planning.  So  it  becomes  able  to  improve  both  the  tech¬ 
nical  system  and  the  working  conditions  and  is  always  informed  about  the  expected 
stress  and  potential  risks  in  combination  with  the  economic  data. 

Conclusions 

With  the  presented  approach  we  intended  to  show  new  application  fields  for  simulation 
models.  Additional  knowledge  is  integrated  into  the  additional  modules.  Results  are  pre- 
processed  for  the  user.  Of  course,  the  user  has  to  draw  his  own  conclusions  and  take  his 
own  decisions  as  ever.  Simulation  is  not  able  to  optimize  the  system,  but  is  able  to  evalu¬ 
ate  planning.  The  supporting  level  for  the  planning  team  is  increasing. 

The  shown  modular  approach  concerning  the  post-run  data  processing  allows  to  use  clas¬ 
sical  simulation  tools  under  various  conditions  in  the  diverse  fields  of  application. 
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Abstract 

Multimedia  communication  is  a  technology  of  growing  importance  for  user  interfaces  in 
automation  and  process  control.  We  are  investigating  and  developing  methods  and  utili¬ 
ties,  that  provide  an  object-oriented  integration  of  multimedia  aspects  into  modem  dis¬ 
tributed  process  control  systems.  A  general  approach  of  a  multimedia-based  teleservice- 
concept  will  be  proposed.  We  introduce  an  object-oriented  multimedia  communication 
system  based  on  our  implementation  of  the  CORBA  standard,  which  will  also  be  de¬ 
scribed  briefly.  A  first  prototype  of  the  communication  system  allows  a  periodic  ex¬ 
change  of  stiU  images  between  a  supervising  camera  at  one  computer  and  a  remote  com¬ 
puter  via  a  small-band  ISDN  telephone  line. 

Introduction 

Multimedia  conununication  is  a  technology  of  growing  importance  for  user  interfaces  in 
automation  and  process  control.  Some  actual  keywords  related  to  multimedia  are  tele¬ 
working,  teleleaming  and  teleservice,  and  these  also  influence  concepts  of  process  control 
engineering.  Think  of  acoustic  and  visual  data  obtained  from  microphones  or  cameras 
applied  to  a  certain  automation  system.  Simply  used  and  understood  as  a  new  brand  of 
sensors  or  actors  they  can  improve  automation  concepts  significantly.  Supervising  and 
monitoring  of  certain  process  states  is  simplified,  if  the  operator  can  "see"  and  "hear"  the 
real  process  at  the  same  control  station,  where  the  process  is  visualized.  The  user  inter¬ 
face  is  much  more  intuitive,  if  the  operator  can  interact  on  images  of  the  real  things  (for 
example  switch  on/off  a  motor  with  a  virtual  button  on  a  real  image).  A  large  impact  and 
added  value  can  be  expected  fi'om  the  integration  of  multimedia  concepts  in  teleservice 
applications.  The  mean  time  for  service  and  diagnosis  of  complex  machines,  installed  far 
away  from  the  supplier,  can  be  significantly  reduced  by  an  intelligent  teleservice  concept. 

We  are  investigating  and  developing  methods  and  utilities,  that  provide  an  object- 
oriented  integration  of  multimedia  aspects  into  modem  distributed  process  control  sys¬ 
tems.  The  main  fundament  is  a  strong  and  efficient  concept  of  object-oriented  communi¬ 
cation.  Based  on  the  CORBA- standard  we  have  developed  an  object  management  system, 
the  MiniORB,  which  promises  for  this.  Another  goal,  which  speaks  for  an  implementa¬ 
tion  of  multimedia  concepts  into  automation  systems,  is  cost  efficiency,  i.e.  low-band¬ 
width  communication  lines,  efficient  data  compression  and  low-cost  equipment.  We  will 
describe  our  experiments  with  a  simple  object-oriented  multimedia  communication  sys¬ 
tem  based  on  our  implementation  of  the  CORBA  standard.  This  prototype  allows  a  peri¬ 
odic  exchange  of  still  images  between  a  supervising  camera  at  one  computer  and  a  re¬ 
mote  computer  via  a  small-band  ISDN  phone  line  with  adequate  quahty  and  speed. 

The  object-oriented  approach 

In  control  engineering  a  hierarchical  concept  of  four  layers  is  generally  used  (Tab.  1). 
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Each  different  layer  has  its  own  responsibilities.  It  defines  different  types  of  interacting 
processes  and  the  necessary  communication  between  them,  concerning  human  users  and 
machines.  AU  participants  can  be  modeled  as  objects.  This  matches  obviously  for  sensors 
and  actors  in  the  process  field,  as  well  as  for  any  computing  system.  Even  all  the  different 
processes  can  be  seen  as  communicating  objects.  Especially  the  human-computer- 
interaction  consists  of  interacting  objects,  but  with  different  levels  of  detail,  depending  on 
the  model  layer,  where  they  live. 


Tab.  1 :  Levels  of  control 


Factory 

administration 

Administrative  functions,  i.e.  in  managerial-economics  and 
marketing  (administrative  management) 

Production  control 

Logistics  control,  quality  control,  controlling,  operations 
scheduling  (economical,  technical  and  disposing  tasks) 

Process  control 

Plant  control  and  supervisory,  machine  control 

0  supervision,  starting  and  stopping  of  processes 
(operative  tasks) 

1  measurement,  command,  control,  visualization 
(elementary  functions  of  process  control) 

Field 

Sensors,  actors,  wires 

(data  acquisition  and  set  point  control) 

The  object-modeling-concept  can  be  explained  with  the  following  example:  Figure  I 
shows  a  part  of  a  chemical  process.  We  can  derive  three  different  classes  of  objects,  con¬ 
tainer,  valve  and  reactor.  The  reactor  looks  like  a  specialized  container  with  a  heater  and 
a  mixer.  Valves  are  of  types  input  control  and  output  control.  Obviously  there  are  hierar¬ 
chical  relations.  The  whole  technical  process  can  be  abstracted  step  by  step  until  the  ob¬ 
ject-model  is  complete. 


Container 


Inputvalvel 


Container! 


InputValve! 


Reaktor 


OutpirtVatve 


CiassStirringCaldron 


Figure  1 :  A  chemical  process  and  its  abstraction 


Considering  an  object-model,  which  is  universal  and  common  for  all  layers  we  further 
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deal  with  abstractions  of  all  different  real-world  objects  and  comnaunication.  Therefor  we 
mark  several  kinds  of  communication-media  (i.e.  speech,  text,  images),  each  of  them 
instantiated  as  objects  from  different  classes  (i.e.  reports,  tables  or  process  values).  The 
communication-media  are  transmitted  via  special  communication-channels  like 
ETHERNET,  field  bus  or  telephone  lines,  according  to  special  communication- 
protocols.  A  universal  model  must  take  this  into  account  and  provide  the  necessary 
mechanisms  and  concepts. 

How  does  the  object-model  match  a  distributed  automation  system?  A  mechanism  is  nec¬ 
essary,  which,  based  on  a  unique  protocol,  provides  a  network-transparent  communica¬ 
tion  between  principally  heterogeneous  computing  platforms.  An  object  should  not  need 
to  have  any  knowledge  about  its  computing  environment,  nor  should  it  know,  whether 
another  object  it  communicates  with,  is  local  or  remote.  Also  a  fixed  client-server  relation 
is  not  possible,  because  the  role  of  client  and  server  can  change  in  the  context  of  an  on¬ 
going  process.  This  demands  a  object-  and  communication  manager  -  a  broker. 

Several  broker  concepts  were  introduced.  OLE  (object  Unking  and  embedding)  and 
CORB  A  (common  object  request  broker  architecture)  are  the  most  important.  OLE  uses  a 
fixed  and  binary  compatible  protocol,  i.e.  it  demands  equal  operating  systems  and  com¬ 
puting  platforms.  The  implementation  of  a  CORBA  object  broker  is  platform  dependant, 
but  the  protocol  is  dear-text  coded  and  independent  of  any  hardware.  This  guarantees 
interoperability  in  any  heterogeneous  distributed  computing  system.  Due  to  these  evident 
advantages  we  chose  CORBA  as  the  basis  for  our  developments. 


A  communication  concept  for  distributed  objects 

The  CORBA  specification  is  part  of  the  "Object  Management  Architecture"  (OMA)  de¬ 
veloped  by  the  OMG^  and  describes,  how  an  "Object  Request  Broker"  (ORB)  has  to  be 
built.  The  ORB  is  the  central  component  within  an  OMA  environment.  It  supervises  and 


^OMG  =  Object  Management  Group 


310 


manages  any  messages  between  the  distributed  objects.  The  architecture  of  a  CORBA 
ORB  is  shown  in  Figure  2. 

A  communication  happens  always  between  two  members:  a  client  object  (CO),  that  needs 
a  service  and  an  object  implementation  (01)  that  offers  a  service.  It  is  neither  platform-, 
nor  implementation-dependent.  The  client  object  demands  a  certain  service  with  a  mes¬ 
sage  called  "request",  that  it  sends  to  the  ORB.  The  ORB  knows,  which  object  imple¬ 
mentation  is  able  to  satisfy  the  client  object’s  demands,  locates  the  01  within  the  network 
and  forwards  the  request  message  to  it.  The  object  implementation  processes  the  request 
and  produces  some  results,  which  it  puts  into  a  "reply" -message,  that  is  then  sent  the  way 
back  to  the  client  object. 

The  MiniORB 

The  MiniORB  is  our  implementation  of  a  CORBA-ORB  (Heinz,  1996).  Divergently  from 
the  original  specification  we  have  not  yet  implemented  dynarmc  requests  and  a  CORBA 
conformant  exception  handling,  i.e.  the  dynamic  invocation  interface  and  the  diverse  re¬ 
positories  (Figure  2)  are  object  of  our  ongoing  work.  The  MiniORB  is  actually  imple¬ 
mented  in  C-i-i-  under  Unix,  the  interprocess  communication  between  any  hosts  is  based 
on  Berkeley  sockets. 
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Figure  3  :  Components  of  the  MiniORB 


Figure  3  shows  the  components  of  the  MiniORB,  It  is  a  client-  and  object  implementa¬ 
tion-  resident  ORB,  i.e.  the  ORB-functions  are  realized  as  library  functions,  which  are 
linked  to  the  respective  code  of  the  client  object  and  the  object  implementation.  These 
libraries  contain 

0  the  ORB-core,  that  implements  the  inter-process  communication  (IPC)  be¬ 
tween  the  distributed  computing  systems,  where  the  communicating  objects 
reside  on 

1  the  ORB-interface,  that  offers  some  general  services  to  the  client  object  and 
the  object  implementation 
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2  the  object  adapter,  that  coordinates  the  cooperation  between  the  object  imple¬ 
mentation  and  the  ORB. 

The  components  "stub"  and  "skeleton"  are  generative  components,  which  originate  from 
the  interface  specification  between  a  client  object  and  the  respective  object  implementa¬ 
tion.  They  are  generated  as  code-appendixes  for  any  new  object  (see  Figure  4).  A  client 
object  is  directly  connected  to  a  local  "proxy  object"  (the  stub),  with  which  it  exchanges 
any  messages.  The  main  task  of  a  proxy  object  is  to  encode  or  decode  the  passing  mes¬ 
sages  to  or  from  the  locally  implemented  ORB.  A  server-agent  task  (not  shown  here),  re¬ 
siding  on  the  remote  system,  resolves  demanded  object-to-object  paths,  initializes  the  cor¬ 
rect  object  implementations  and  activates  the  inter-process  ORB-to-ORB  communication. 
The  remote  ORB  then  exchanges  the  message  with  the  selected  object  implementation. 
The  object  implementation  contains  a  special  skeleton  method,  that  decodes  the  received 
encoded  data  and  calls  the  demanded  service.  The  results  are  transmitted  back  to  the  cli¬ 
ent  object  by  the  opposite  way. 

The  definition  of  interfaces 

The  interfaces  between  client  objects  and  object  implementations  within  a  CORBA- 
environment  are  always  well  defined.  The  communication  between  any  objects  is  trans¬ 
parent,  i.e.  an  object  can  never  determine,  if  its  partner  is  a  remote  or  local  object.  This  is 
provided  by  a  special  interface  definition  language  (IDL),  which  allows  a  neutral  defini¬ 
tion  of  all  services  a  certain  object  implementation  offers.  The  interface  definition  lan¬ 
guage  is  part  of  the  CORBA  specification. 

The  IDL  description  of  an  object  implementation  interface  consists  of  the  following  com¬ 
ponents: 

0  the  name  of  the  interface 

1  the  name  of  all  services  the  object  implementation  offers 

2  the  data  that  are  necessary  for  the  execution  of  each  service 

3  the  data  that  results  from  the  execution  of  each  service 

4  the  needed  data  types 

IDL  is  a  purely  declarative  language.  The  implementation  of  an  IDL-defined  interface 
requires  the  use  of  a  conventional  programming  language.  Therefore  a  mapping  of  the 
IDL  constructs  to  the  constructs  of  the  implementation  language  is  necessary.  The 
CORBA-specification  contains  mapping  rules  for  the  languages  C,  and  Smalltalk. 
We  have  developed  an  IDL-compiler,  which  automatically  generates  a  C-i-f-implemen- 
tation  from  IDL-defined  interfaces  (Sixtus,  1997). 

The  development  of  a  distributed  multimedia-application 

With  the  MiniORB  and  our  IDL  compiler  it  is  quite  simple  to  develop  distributed  appli¬ 
cations:  For  each  certain  object  implementation  an  interface  has  to  be  declared  in  IDL. 
From  each  of  these  interfaces  the  IDL  compiler  automatically  generates  four  sets  of  C-I-+- 
files,  each  consisting  of  a  header-  and  a  implementation-file  {Figure  4): 


0  a  global  file  set,  that  contains  the  implementation  of  code  relevant  for  both  the 
client  object  and  die  object  implementation 

1  a  stub  file  set,  that  contains  the  implementation  of  the  proxy  object 

2  a  skeleton  file  set,  that  contains  the  implementation  of  the  skeleton  methods 

3  the  object  implementation  file  set,  that  frames  the  real  implementation  of  the 
services  the  object  implementation  provides 

The  several  header  files  include  each  other. 


Figure  4 :  The  development  of  a  distributed  environment 


The  developer  has  to  add  the  implementation  code  for  any  services  the  object  implemen¬ 
tation  provides  to  the  implementation  file  set.  Furthermore  he  has  to  write  a  server  pro¬ 
gram  that  activates  the  object  implementation  at  runtime  and  a  client  program  which  calls 
the  services  of  the  object  implementation  via  the  proxy  object.  After  all  the  source  code  is 
compiled  and  linked,  the  client  object  can  transparently  communicate  to  the  object  im¬ 
plementation,  regardless  where  this  resides  in  the  distributed  system. 


Figure  5 :  First-step  prototype  of  a  teleservice  system 
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The  prototype 

Our  first  prototype  implementation  of  a  distributed  multimedia  system  for  process  control 
and  teleservice  applications  is  shown  in  Figure  5.  It  is  a  low  cost  communication  system 
based  on  our  MiniORB,  that  provides  the  ability  to  exchange  image  data  from  a  camera 
between  a  local  and  a  remote  computer  via  a  simple  ISDN  telephone  line.  We  use  a  Con- 
nectix  QuickCam™,  which  is  wide  spread  in  low-cost  video  applications  because  it  works 
with  any  common  Centronics  parallel-port.  Beside  the  CORBA  standard  we  also  use 
shared-memory  and  semaphore  concepts  for  inter-process  communication.  The  comput¬ 
ing  platform  of  our  prototype  are  personal  computers  with  a  LINUX  operating  system. 

We  evaluated  the  three  parts  of  transmission,  i.e.  image  acquisition,  data  communication 
and  image  presentation,  to  find  out  the  overall  possible  performance  (AmshofT,  1997). 
Evaluation  concerned  processing  speed,  image  resolution  and  size,  image  compression 
and  refresh  rates.  We  use  JPEG  compression/decompression  and  found  a  compression 
factor  of  60  to  be  fully  satisfying  for  diagnostic  or  monitoring  tasks.  The  following  tables 
show  some  results. 

Tab.  2  shows,  that  with  our  environment  an  average  image  flow  of  1/2  image/s  is  possi¬ 
ble  (including  compression),  with  a  maximum  acquisition  speed  of  193  kBytes/s,  This 
can  be  eventually  advanced  by  using  the  ECP/EPP  transmission  mode  of  modem  parallel 
ports,  which  promises  a  speedup-factor  of  5..  10. 


Tab.  2  :  Image  acquisition,  Size  320^240  pixel  (full  color),  JPEG  quality -factor  60 


486  DX  66 
unidirectional 
parallel  port 

Pentium  133 
unidirectional 
parallel  port 

Pentium  133 
bi-directional 
parallel  port 

Compressed-image  size 

10100 

8912 

9082 

Delay  before  first  image 

6.03  s 

2.32  s 

1.6  s 

Time  between  two  images 

4.37  s 

1.88  s 

1.19  s 

Tab.  3  shows,  that  an  average  communication  speed  of  about  5900  Bytes/s  is  possible 
with  our  MiniORB.  This  is  slightly  below  the  approximate  maximum  of  6500  Bytes/s  on 
a  ISDN  B-channel,  due  to  the  communication  overhead  caused  by  the  object  manage¬ 
ment. 


Tab.  3 :  Image  transmission  performance 


File  size 

Transmission  time 

Performance 

(Byte) 

(s) 

(Bytes/s) 

1992  Byte 

0.345 

5773 

4009 

0.676 

5930 

5998 

1.007 

5956 

8006 

1.340 

5974 
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Tab,  4  represents  the  image  presentation  including  decompression  and  local  communica¬ 
tion.  It  shows,  that  with  our  prototype  a  frame  rate  of  5  images/s  is  simply  realizable. 

Tab,  4 :  Image  presentation,  Size  320*240  pixel,  JPEG  quality-factor  60  _ 


- - - O''  r- - ..r- 

- £ - - * - 

486DX  66 
lMB/16  Bit  Graphics 
Subsystem 

Pentium  133 

2MB/16  Graphics 
Subsystem 

Resolution 

1024  *  768  pixel 

Image  size 

12637  Byte 

Time  between  two  images 

918  ms 

191  ms 

Resolution 

640  *  480  pixel 

Image  size 

12637  Byte 

Time  between  two  images 

860  ms 

182  ms 

Conclusions 

The  concept  of  an  object-oriented  multimedia  communication  system  for  teleservice  ap¬ 
plications  was  introduced.  We  presented  the  state  of  development  of  our  object-oriented 
environment,  the  MiniORB  and  the  IDL-compiler.  It  was  shown,  that  the  grade  of  inte¬ 
gration  of  video  (and  audio)  data  into  a  process  control  system  should  depend  on  the  real¬ 
time  demands  of  the  process.  A  regular  ISDN-connection  (one  channel,  64  KBit)  allows  a 
periodical  exchange  of  still  images  and  packed  audio  data  every  few  seconds.  This  is  suf¬ 
ficient  for  many  teleservice  applications.  Online  process  monitoring  with  audio-  and 
video-  components  or  teleconferencing  requires  higher  bandwidths.  But,  for  example,  the 
online  diagnosis  of  erroneous  parts  of  a  manufacturing  machine  works  fine  with  low  im¬ 
age  rates  or  still  images. 

The  next  step  of  evaluation  will  be  the  transmission  of  audio  data  and  process  values 
between  a  technical  process  and  its  control  system  via  the  same  communication  channel 
at  the  same  time,  eventually  by  using  a  second  ISDN  B-channel.  A  main  focus  of  interest 
will  be  on  the  synchronization  of  audio  and  video  streams  within  our  object  management 
system. 
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Abstract 

With  the  growth  of  the  Internet  the  ability  to  share  remote  laboratory  resources  and  to 
monitor  and  control  remote  devices  becomes  more  and  more  important.  Since  the  Inter¬ 
net  makes  no  guarantees  of  the  communication  link’s  bandwidth,  tele-operation  systems 
have  to  deal  with  possible  lags  in  communication.  This  paper  describes  the  architecture  of 
the  ‘  ‘Robotic  Tele  Lab”,  a  tele-operation  system  allowing  multiple  researchers  in  differ¬ 
ent  locations  to  observe  experiments  on  our  task-level  controlled  mobile  robot  RHINO 
over  the  Internet.  To  cope  with  the  poor  communication  properties  of  the  Internet,  our 
system  uses  a  distributed  virtual  environment,  predictive  simulations  and  IP-Multicasting. 
This  approach  allows  the  monitoring  of  complex  experiments  even  over  low  bandwidth 
connections  with  varying  transmission  delays.  We  further  describe  an  experiment  with 
our  task-level  controlled  mobile  robot  RHINO.  During  this  experiment  RHINO  operated 
for  six  days  as  a  tourguide  in  a  populated  museum  and  was  tele-operated  over  the  Internet 
by  more  than  600  virtual  visitors. 

1  Introduction 

A  well  known  application  area  for  robots,  and  especially  tele-operated  autonomous  ro¬ 
bots,  is  the  exploration  of  dangerous  environments  or  of  environments  whose  direct 
observation  is  in  some  way  difficult.  The  mars  pathfinder  mission  [Sto96]  is  perhaps  the 
most  popular  example  of  this  type  of  application.  However,  we  are  concerned  with  a  dif¬ 
ferent  research  direction  in  autonomous  robotics,  namely  the  design  of  service  robot 
systems.  Such  are  robots  capable  of  carrying  out  tasks  like  delivery  jobs  autonomously  in 
ordinary  environments  like  hospitals  and  offices.  It  is  a  common  occurrence  that  these 
systems  are  being  jointly  developed  by  researchers  working  in  different  countries.  Test¬ 
ing  new  robot  control  systems  is  a  time  consuming  and  expensive  process  under  these 
circumstances  because  the  developers  have  to  meet  in  one  location  and  have  to  assemble 
the  whole  system  before  they  can  test  it. 

This  paper  introduces  the  “Robotic  Tele  Lab”  (RTL),  a  tele-operation  system  allowing 
researchers  in  different  locations  to  test  and  demonstrate  new  control  software  on  a  robot 
over  the  Internet.  We  believe  that  this  system  will  help  to  improve  the  utilisation  of  ex¬ 
pensive  robots  and  will  lead  to  a  more  efficient  development  process.  Software 
components  can  be  tested  frequently  and  new  results  can  be  demonstrated  without  the 
need  to  travel. 

RTL  is  different  in  many  respects  from  tele-operation  interfaces  used  for  exploration 
tasks.  The  operating  environment  is  almost  known  but  changes  dynamically  and  since 
RTL  is  using  the  Internet  as  the  communication  Hnk,  it  has  to  cope  with  low  bandwidth 
and  varying  communication  delays.  Traditional  methods  of  tele-operation,  such  as  video 
transmissions  and  direct  remote  control  of  robots,  are  not  feasible  under  these  circum- 


317 


stances.  We  use  the  task-level  control  system  of  our  mobile  robot  RHINO  and  a  3D  com¬ 
puter  graphics  visualisation  of  the  operating  environment  instead,  A  sophisticated  scheme 
of  synchronised  simulations,  adopted  from  the  field  of  distributed  interactive  simulation 
[MBZ95-I-],  bridges  transnussion  delays  and  helps  to  further  reduce  bandwidth  require¬ 
ments.  For  the  transmission  of  state  changes  we  employ  IP-Multicasting,  the  most 
efficient  method  for  transferring  information  to  many  destinations  over  the  Internet.  As 
the  robot’s  environment  is  changing  dynamically  (people  walk  around,  doors  are 
opened/closed,  etc.),  these  changes  in  the  real  world  have  to  be  integrated  into  the  world 
model,  which  forms  the  basis  of  the  3D  visualisation.  RTL  can  be  easily  extended  by 
specialised  sensor  interpretation  modules,  each  of  them  performing  particular  subtasks  as 
part  of  the  overall  system  goal. 

The  strength  of  these  concepts  has  been  demonstrated  during  a  museum  tourguide  proj¬ 
ect.  During  this  experiment,  RHINO  operated  for  six  days  as  a  tourguide  in  a  populated 
museum  and  was  tele- operated  over  the  Internet  by  more  than  600  virtual  visitors.  The 
Web  interface  of  the  tourguide  system  was  the  first  application  of  RTL  technology,  which 
will  be  introduced  in  the  second  half  of  this  paper.  Due  to  limitations  of  the  Internet 
communication  facilities  of  the  WWW,  some  differences  between  the  RTL  architecture 
and  the  tourguide  interface  exist  and  these  will  also  be  explained. 

2  Related  Work 

So  far,  research  on  tele-operated  autonomous  robots  has  focused  mainly  on  robots  ex¬ 
ploring  dangerous  environments  [HHF+95,  FPW-i-95]  and  on  the  design  of  autonomous 
robot  systems  for  space  missions  [HBJJ94,  Sto96].  In  contrast  to  RTL,  these  systems  use 
leased  tines  and  satellite  links  for  communication.  High  bandwidth  is  permanently  avail¬ 
able  and  transmission  delays  are  nearly  constant.  Visualisation  is  carried  out  mainly  by 
video  transmission  and  is  supported  by  3D  computer  graphics  to  bridge  the  veiy  large 
transmission  delays  of  up  to  several  minutes.  The  environment  of  the  robot  is  assumed  to 
be  static  in  these  systems. 

The  Internet  communication  architecture  of  RTL  is  based  on  NPSNET-IV  [MBZ-i-95],  a 
distributed,  interactive  virtual  3D  multi-user  environment,  connecting  different  training 
simulators  over  the  Internet,  From  the  NPSNET-IV  system  RTL  adopts  the  use  synchro¬ 
nous  predictive  simulations  and  the  use  of  IP-Multicasting.  NPSNET-IV  is  a  pure  virtual 
environment.  The  RTL  system  on  the  other  hand  is  designed  to  keep  its  virtual  environ¬ 
ment  synchronised  with  the  actual  environment  of  a  real  robot. 

3  Components  of  RTL 

From  an  abstract  point  of  view,  RTL  has  a  client-server  architecture.  The  server  resides 
on  a  computer  in  the  robotics  laboratory  and  its  task  is  to  integrate  all  information  avail¬ 
able  from  the  robot  (e.g.  position,  velocity,  state  of  all  actuators  and  sensor  data)  into  its 
internal  world  model.  Significant  changes  in  the  state  of  the  world  will  be  transferred  to 
all  connected  RTL  clients.  The  clients  will  update  their  own  copy  of  the  world  model  ac¬ 
cording  to  the  incoming  state  information  and  visualise  the  actual  laboratory  environment 
using  3D  computer  graphics.  In  the  following  sections  we  will  briefly  explain  the  main 
components  of  the  RTL  system:  Internet  communication,  clients,  servers  and  the  syn¬ 
chronisation  of  the  virtual  robot  with  the  real  robot.  Please  note  that  actual  control  of  the 
robot  is  not  part  of  the  system  itself  because  its  main  purpose  is  to  test  and  demonstrate 
new  robot  control  software. 
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3 A  Internet  Communication 

A  big  difference  between  RTL  and  typical  client-server  systems,  for  example  the  'WWW, 
is  the  network  communication  technology  used.  In  typical  client-server  systems,  the 
server  sends  information  to  a  single  client  only  on  request.  The  RTL  server  broadcasts 
state  changes  to  all  connected  clients  at  once.  Broadcasting  is  achieved  using  IP- 
Multicasting  over  the  multicast  backbone  (MB one)  [Dee89]  of  the  Internet.  Clients  reg¬ 
ister  at  the  so-called  multicast  address  of  the  server.  Together  the  server  and  clients  form 
a  multicast  group.  In  multicast  groups  every  message  sent  by  one  group  member  is  re¬ 
ceived  by  all  the  other  group  members.  The  use  of  IP-Multicasting  has  at  least  two 
advantages. 

1.  The  communication  expenditure  of  the  server  is  independent  of  the  number  of  clients. 
For  this  reason,  the  architecture  is  scalable  to  a  large  number  of  clients  listening  to  the 
server  at  the  same  time. 

2.  Since  the  clients  register  at  a  multicast  address  and  not  at  the  server  directly,  the 
server  can  be  split  up  into  separate  modules,  each  handling  distinct  parts  of  the  com¬ 
plex  world  update  task. 

3,2  Clients  and  Servers 

We  will  now  describe  the  basic  building  blocks  of  RTL  clients  and  servers:  the  world 
database,  the  predictive  simulations,  the  visualisations  on  the  client  side  and  the  synchro¬ 
nisation  of  the  virtual  robot  with  the  real  robot.  In  addition  we  will  explain  how  they 
interact. 


Display 


Experiment 


Figure  1.1:  Block  diagram  of  an  RTL  client 


Figure  1.2:  Block  diagram  of  an  RTL  server 


World  database:  The  central  part  of  the  RTL  server  and  the  chents  is  the  world  data¬ 
base.  This  object  oriented  database  contains  the  3D  world  model  of  the  RTL  system.  The 
same  model  is  used  for  the  clients  and  the  servers.  Every  object  in  this  model  refers  to  a 
real  object  in  the  operating  environment,  for  example  a  piece  of  furniture  or  RHINO  it¬ 
self.  It  is  the  job  of  the  RTL  server  to  integrate  changes  in  the  state  of  the  world  into  the 
database.  These  changes  are  either  recognised  by  dedicated  sensor  interpretation  mod¬ 
ules,  for  example  in  the  case  where  a  door  has  been  closed,  or  are  announced  directly  by 
the  object  whose  state  has  changed.  One  example  of  this  type  is  the  robot’s  navigation 
system.  Every  object  inside  the  server’s  database  contains  a  method  which  transfers 
changes  of  its  attribute  values  to  its  twin  objects  inside  the  clients’  databases.  This  way 
all  databases  will  agree  upon  their  state  over  time.  Internet  communication  is  completely 
hidden  from  the  remaining  parts  of  the  system. 
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The  world  database  distinguishes  active  and  passive  objects.  Passive  objects  are  furniture, 
doors  and  other  parts  of  the  world  which  do  not  change  their  state  by  themselves.  Attrib¬ 
ute  values  of  passive  objects  are  only  changed  by  the  sensor  interpretation  modules  every 
time  they  detect  a  change  of  the  real  object’s  state.  Every  change  is  transferred  to  all  twin 
objects  in  the  clients*  databases. 

Examples  of  active  objects  are  the  robot  itself  and  humans.  In  contrast  to  passive  objects 
active  objects  change  their  states  by  themselves.  Inside  the  world  database  these  changes 
are  carried  out  by  special  simulation  procedures,  one  for  each  type  of  active  object. 

Simulation:  A  simulation  procedure  changes  the  state  of  the  active  object,  every  few 
milliseconds.  Simulation  takes  place  in  the  server’s  database  as  well  as  in  the  clients’  da¬ 
tabases.  The  simulation  fulfils  two  purposes.  (1)  It  is  able  to  predict  the  behaviour  of  the 
robot  for  several  seconds,  thus  bridging  the  varying  transmission  delays  of  the  Internet. 
(2)  As  simulation  takes  place  inside  the  server  and  inside  the  clients,  database  updates  for 
active  objects  are  only  sent  if  the  predicted  state  of  the  object  differs  significantly  from 
the  real  state  of  the  object.  This  dead  reckoning  simulation  keeps  all  the  databases  nearly 
synchronised  over  time  and  it  also  considerably  reduces  the  bandwidth  requirements  of 
the  system.  Watching  the  robot  move  in  a  static  environment  produces  a  maximum  Inter¬ 
net  traffic  of  only  480  bytes  per  second  and  the  average  traffic  is  less  than  200  bytes  per 
second. 


Figure  2:  The  2D  visualisation  showing 
the  robot’s  position 


Visualisation:  An  RTL  client  contains  a  2D  and  a  3D  visualisation  of  the  laboratory  en¬ 
vironment.  These  two  visualisations  form  the  tele-presence  interface  of  the  RTL  system. 
The  2D  visualisation  shows  a  top  view  of  the  whole  operating  environment  including  the 
actual  position  of  the  robot  (fig.  2).  Since  a  robot  marker  is  animated  by  the  simulation 
according  to  the  movements  of  the  real  robot,  the  user  is  able  to  observe  the  real  robot’s 
trajectory. 

Using  an  OpenGL  based  3D  visualisation,  users  of  the  real  robot  can  define  virtual  cam¬ 
eras,  enabling  them  to  view  the  virtual  environment  from  various  perspectives  (fig.  3).  A 
virtual  camera  can  also  be  bound  to  the  position  of  one  of  the  robot’s  real  cameras  (fig.  4). 
Since  the  simulations  are  synchronised,  the  user  will  see  the  virtual  environment  from  the 
perspective  of  the  real  robot,  even  while  it  is  moving. 
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Figure  3.1:  Virtual  view  of  the 

Figure  3.2:  Real  view  of  nearly 

robot 

the  same  position 

Synchronisation:  In  this  paragraph  we  will  briefly  explain  how  the  RTL  server  is  con¬ 
nected  to  the  RHINO  system  [BBC95-I-,  TBB-J-]  and  how  the  virtual  robot’s  position  is 
synchronised  with  RHINO’S  position.  The  RHINO  system  consists  of  the  robot  itself,  an 
RWI B21  robot,  and  several  software  modules,  each  performing  a  specific  navigation  or 
planning  task.  The  modules  exchange  information  asynchronously  via  TCP/IP  using  the 
message  passing  library  TCX  [Fed93].  Two  modules  are  employed  for  the  synchronisa¬ 
tion  tasks:-  (1)  the  navigation  and  collision  avoidance  module  [FBT97]  and  (2)  the 
localisation  module  [BFHS96].  The  navigation  module  sends  robot  state  information,  po¬ 
sition,  velocities  and  accelerations,  to  the  RTL  server  up  to  4  times  a  second.  These 
values  are  used  by  the  RHINO  interface  of  the  server  to  re-initialise  the  simulation  if  the 
simulated  values  differ  significantly  from  the  real  values.  However,  due  to  dead  reckon¬ 
ing  errors  of  the  robot,  the  position  information  of  the  navigation  module  may  not  be 
correct.  It  is  the  task  of  the  localisation  module  to  find  the  robot’s  absolute  position  and 
to  keep  track  of  it.  It  is  accurate  to  within  10  square  cm.  The  RHINO  interface  of  the 
RTL  server  uses  correction  parameters  sent  by  the  localisation  module  to  compensate  for 
the  dead  reckoning  error  of  ihe  navigation  module. 


As  the  RTL  system  is  still  in  its  early  stages,  only  a  first  prototype  implementation  of  the 
system  as  described  above  exists.  We  only  carried  out  some  small  tests  over  our  local 
department  network.  Although  the  system  gave  us  a  good  impression  about  what  was 
really  going  on  in  the  laboratory,  these  tests  can  not  be  compared  to  tele-operation  over 
the  Internet,  because  network  communication  is  much  better  over  a  local  area  network. 
However,  the  museum  tourguide  project  proved  that  the  underlying  concepts  scale  to  the 
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Internet  as  the  communication  link  for  tele-operation  systems.  Before  we  go  on  to  de¬ 
scribe  this  project,  we  want  to  suggest  another  possible  application  of  the  RTL  system. 

3J  Simulation  of  multi-robot  experiments 

We  want  to  explain  the  use  of  RTL  as  a  simulation  environment  for  multi-robot  experi¬ 
ments.  An  RTL  server  can  be  used  as  a  pure  RHINO  simulator.  In  this  configuration  the 
modules  of  the  RHINO  system  are  connected  to  the  simulation  inside  the  RTL  server  in¬ 
stead  of  to  the  real  robot.  As  the  world  database  is  not  restricted  to  only  one  robot,  several 
servers  can  be  started  as  simulators,  thus  leading  to  a  multi-robot  simulation  environment. 
Please  note  that  every  simulated  robot  is  able  to  detect  every  other  simulated  robot  with 
its  simulated  sensors,  as  soon  as  it  comes  within  reach.  The  multicasting  based  communi¬ 
cation  architecture  makes  it  easy  to  distribute  the  whole  simulation  environment  over 
several  workstations  to  gain  the  computation  power  needed.  Thus  experiments  with  con¬ 
trol  software  for  co-operating  autonomous  robots  can  be  easily  carried  out. 

For  the  museum  tourguide  project,  a  facility  based  on  RTL’s  world  database  has  been  de¬ 
veloped,  allowing  the  navigation  system  of  the  real  robot  to  detect  objects  with  virtual 
sensors  which  could  not  be  directly  detected  with  its  real  sensors.  Using  this  facility, 
simulated  and  real  robots  can  also  be  used  together  in  one  experiment. 

4  The  museum  tourguide  project 

From  May  29.  to  June  1st  1997  and  on  the  first  two  days  of  July,  RHINO  operated  as  a 
tourguide  in  the  “Deutsches  Museum  Bonn”.  The  goal  of  this  experiment  was  to  demon¬ 
strate  the  potential  of  the  RHINO  system  to  a  large  audience.  Our  tourguide  can  be  used 
in  two  different  modes:  local  mode  and  Internet  mode.  If  it  is  operated  in  local  mode, 
visitors  to  the  museum  can  attend  a  guided  tour  to  selected  exhibits.  The  robot  will  move 
to  the  exhibits,  by-passing  other  visitors,  and  will  play  back  pre-recorded  information 
about  particular  exhibits  when  they  are  reached.  If  the  robot  is  operated  in  Internet  mode, 
WWW  users  are  able  to  send  it  to  one  or  more  of  the  exhibits  listed  on  the  “Tell  RHINO 
where  to  go”  page  (fig.  5)  and  the  tourguide  system  schedules  the  incoming  requests  into 
a  tour  through  the  museum.  Independent  of  the  mode  of  operation,  new  pictures  from 
one  of  the  robot's  cameras,  and  from  a  fixed  camera  observing  the  exhibition,  are  placed 
on  a  WWW  server  every  5  seconds. 

The  WWW  interface  of  the  tourguide  employs  techniques  developed  for  the  RTL  system. 
However  the  communication  architecture  had  to  be  considerably  changed  because  of 
some  deficiencies  of  WWW  technology.  The  main  purpose  of  the  WWW  is  still  to  make 
static  information  available.  Current  Web  browsers  only  have  restricted  facilities  for  dis¬ 
playing  dynamic  Web  pages.  We  will  first  discuss  these  opportunities  before  we  go  on  to 
describe  the  actual  design  of  the  Web  interface  of  the  museum  tourguide. 

Client  pull:  An  annotation  in  the  head  of  the  HTML  page  informs  the  browser,  that  it  has 
to  reload  the  page  after  a  given  time  interval.  However,  it  has  to  be  taken  into  account, 
that  the  server  has  no  possibility  of  synchronising  these  updates  with  external  events  such 
as  the  robot  reaching  an  exhibit. 
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Tell  RHINO  where  to  go 

Please  enter  your  name  or  email  adress  -and^regteter  Itto  monitor  that  your  requests  are  scheduled. 


Tlia  numbers  mark  the  exhibits  of 

\i(/hiqh  RHINO  cari  be  sent 
are  sensitive  to  the  clicking.  : 
allows  you  to  view  the  exhitjjt 
frqilrl  perspectives 

Pleas?  Ch.r  on .  H«  h^v^hPtyk 

According  to  the  requests  of  the  VlrtiMl  wsftWB  RHIISIQ'S  lour  you  tathefoHowing  exhibits 
1.  Airbus  ruddpi^S)for  schulz 


Figure  5:  The  "Tell  RHINO  where  to  go”  page 


Server  push:  This  facility  makes  use  of  multi-part  documents.  A  multi-part  document 
consists  of  the  concatenation  of  a  sequence  of  simple  documents.  A  multi-part  JPEG 
document,  for  example,  consists  of  a  sequence  of  JPEG  images.  The  browser  displays 
every  part  of  a  multi-part  document  as  soon  as  it  arrives.  Using  server  push,  the  server 
has  control  over  the  point  in  time  when  an  update  is  sent  to  the  client.  However,  it  gets 
no  feedback  about  when  the  new  content  has  actually  arrived  at  the  client.  On  slow  Inter¬ 
net  connections  this  may  lead  to  old  information  accumulating  and,  as  a  result,  the  virtual 
visitor  will  get  outdated  information.  We  did  not  use  server  push  for  the  Web  interface  of 
the  tourguide  for  these  reasons, 

Java  Applets:  A  Java  applet  is  a  Java  program,  running  in  a  Web  browser.  Using  Java 
we  are  not  bound  to  proprietary  client  server  communication  facilities,  but  are  able  to  im¬ 
plement  our  own  communication  scheme.  However,  there  are  two  restrictions.  (1)  For 
security  reasons  Java  applets  are  only  allowed  to  contact  the  computer  from  which  they 
have  been  loaded.  (2)  At  the  time  of  the  tourguide  project,  most  Web  browsers  were  not 
capable  of  multicasting.  For  these  reasons  it  was  not  possible  to  simply  adopt  the  com¬ 
munication  scheme  used  by  RTL. 


323 


ti3s>*  if /mm  iste  .  'Htmi 


I 

H; 


.:\"i 

Miiii 


■. 


!,:  v.Kv  ®  j 


Cr  C-nree  refng^ator 


•  .  v.*  V*  • 


Figure  6:  Pictures  from  the  exhibition 


We  employed  client  pull  and  Java  applets  for  the  Web  interface.  Client  pull  is  used  to  re¬ 
load  the  images  on  the  “Online  Picture  Page”  at  regular  intervals.  Together  with  these 
images  a  GIF  picture  is  displayed,  containing  a  top  view  of  the  exhibition  and  the  robot's 
position  at  the  time  of  image  capture  (fig.  6).  Every  time  these  three  images  are  reloaded, 
approximately  ISCKX)  bytes  have  to  be  transmitted  and,  because  of  this,  only  low  update 
rates  are  possible.  The  update  rate  is  individually  configurable  by  the  virtual  visitors  and 
most  of  them  tuned  it  to  30  seconds.  Obviously  it  is  not  possible  to  observe  tlie  robot's 
behaviour  under  these  conditions  and  so  we  implemented  a  simplified  version  of  the  2D 
visualisation  of  an  RTL  client  in  Java  which  could  be  viewed  on  a  separate  Web  page. 
This  applet  achieves  a  smooth  animation  of  the  robot  and  additionally  explains  the  robot's 
actions  in  one  line  of  scrolled  text  (fig.  7).  Instead  of  IP-Multicasting,  the  applet  commu¬ 
nicates  over  a  simple  unicast  UDP  connection  with  the  server. 

More  than  630  different  virtual  visitors  used  this  Web  interface  during  the  experiment. 
They  sent  RHINO  to  1264  target  points.  In  only  three  cases,  was  the  robot  unable  to 
complete  its  job.  In  all  three  cases  failure  was  caused  by  minor  hardware  problems  like 
empty  batteries.  The  large  number  of  completed  jobs  demonstrates  the  high  reliability  of 
the  whole  system.  The  tourguide  experiment  proves  that,  thanks  to  contemporary  AI 
technology,  autonomous  robots  can  safely  be  tele-operated  over  the  Internet  and  even 
over  the  WWW. 


ss  Wctscape;  RHWOt -RMifgufcfc (appiel) 


Find  out  where  I  am  (Java  appictveralon) 

This  is  an  cuOine  of  RHINO’S  environment.  The  numbers  marie  the  esehibits  to  v/hich  RH INO  can  be 
imt.  The  yeflow  circle  Is  RHIN O’s  current  position. 


:-..j  V«™^™.V...5.WA%^. 


Figure  7;  The  Java  applet.  The  trace  of  the  robot’s  trajectory 
shows  the  smoothness  of  the  animation. 


5  Conclusions 

This  paper  introduced  the  “Robotic  Tele  Lab”,  a  tele-operation  system  allowing  re¬ 
searchers  all  over  the  world  to  test  and  demonstrate  robot  control  systems  via  Internet. 
RTL  demonstrates,  that  the  Internet  is  a  suitable  communication  medium  for  tele¬ 
operation  systems,  despite  its  poor  communication  properties.  We  managed  to  overcome 
these  problems  using  the  combination  of  the  three  key  concepts  of  RTL:  a  distributed 
virtual  environment,  synchronised  predictive  simulations  and  a  task-level  controlled  robot 
with  advanced  sensor  interpretation  capabilities. 

Development  of  the  RTL  system  is  still  at  an  early  stage:  work  has  until  now  mainly  been 
focused  on  the  network  communication  and  visualisation  part  of  the  system  and  on  the 
tourguide  interface.  The  environment  is  still  assumed  to  be  static  in  the  current  imple¬ 
mentation,  One  of  the  next  steps  will  therefore  be  the  development  of  sensor 
interpretation  modules.  Modules  which  detect  the  state  of  doors  and  the  position  of  fur¬ 
niture  have  to  developed  as  well  as  a  module,  which  detects  humans  and  computes  their 
walking  direction  and  walking  speed. 

In  addition  we  want  to  extend  the  RTL  system  so  that  different  researchers  can  carry  out 
experiments  with  new  robot  control  software  over  the  Internet  together,  each  researcher 
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checking  their  own  part  of  the  new  software.  For  this  goal  to  be  achieved,  we  want  to  im¬ 
plement  an  access  control  mechanism  on  top  of  the  RHINO  system  because  we  believe, 
that  such  experiments  are  particularly  prone  to  access  errors,  such  as  two  control  compo¬ 
nents  trying  to  drive  the  robot  to  two  different  destinations  at  the  same  time. 
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Abstract 

Because  of  their  relative  simplicity,  ultrasonic  sensors  have  been  widely  used  in  the 
robotics  community  for  measuring  distances.  With  the  development  of  always  more 
accurate  video  cameras  and  powerful  image  processing  systems,  ultrasonic  sensors  have 
become  less  popular.  However,  in  nuclear  applications  for  example,  ultrasonic  sensors 
have  proved  to  resist  very  well  to  radiations  and  are  thus  still  widely  used. 

But  these  sensors  have  a  lot  of  shortcomings  and  sophisticated  algorithms  have  to  be  used 
to  obtain  reliable  results.  We  present  in  this  paper  several  algorithms  that  temporally  filter 
the  data  and  fuse  them  to  build  a  map  of  the  environment. 

Furthermore,  as  video  cameras  cannot  be  used  due  to  poor  resistance  of  electronics 
against  high  level  of  radiations,  an  alternative  solution  is  needed  to  help  the  user  to 
remotely  control  the  robot  or  the  arm  for  manipulation.  In  this  case  a  3D  representation  of 
the  environment  is  needed.  We  propose  here  a  solution  based  on  today  Internet  tools: 
VRML  and  Java.  We  demonstrate  the  application  on  a  mobile  robot  Nomad200, 

Keywords:  robotics,  sensors,  VRML,  Java. 


Introduction 

Maintenance,  repair,  and  dismantling  operations  in  nuclear  facilities  must  be  performed 
remotely  to  minimize  contamination  risks  and  occupational  doses  to  the  operators. 
Sensor-based  teleoperation  enhances  safety,  reliability  and  performance  by  helping  the 
operator  in  difficult  tasks  when  remote  perception  is  difficult.  Mobile  robots  can  be  used 
to  explore  contaminated  sites  for  maintenance  purposes  or  after  an  incident. 

One  major  problem  is  the  poor  resistance  of  common  sensors  to  radiations.  The  video 
cameras  generally  used  in  robotics  applications  do  not  resist  very  well  to  high  radiation 
doses.  In  this  case,  special  radiation-hardened  sensors  have  to  be  used:  specific  ultrasonic, 
force  and  optical  fiber  optics  sensors  have  been  studied  and  developed  for  many  years. [1] 
Ultrasonic  sensors  are  used  to  detect  obstacles  and  to  measure  distances;  they  can  be  used 
on  a  mobile  robot  or  on  a  teleoperated  arm.  They  offer  a  good  resistance  in  such 
conditions  but  they  return  very  limited  information,  such  as  distance  to  a  point,  unlike 
video  cameras  which  deliver  high-level  very  comprehensive  (for  a  human)  information. 
Ultrasonic  sensors  have  a  relatively  good  precision  but  offer  a  lot  of  uncertainties  and 
we  need  intelligent  data  processing  to  extract  useful  information. 

When  looking  at  the  raw  data  acquired  during  the  motion  of  a  mobile  robot,  we  can  see 
that  some  measurements  are  completely  wrong.  It  is  difficult  to  detect  these  erroneous 
values  when  the  data  are  spatially  comWed,  but  if  a  temporal  sequence  is  recorded,  we 
can  more  easily  find  and  reject  it. 
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In  our  laboratory,  we  use  an  electrical  mobile  robot  Nomad200  to  develop  and  test  the 
algorithms.  This  system  can  be  controlled  remotely  via  a  HP-UNIX  or  PC-Linux 
workstation.  The  data  acquired  by  the  sensors  can  be  viewed  in  different  windows  but  the 
user  can  only  see  a  2D  representation  of  the  world.  This  is  perhaps  enough  in  a  well- 
known  laboratory  but  not  in  a  real  world.  We  have  then  developed  a  3D  virtual 
representation  of  the  robot  and  of  its  environment.  This  application  is  based  on  the  new 
tools  offered  by  the  Internet  technology  VRML  and  Java. 

We  will  begin  with  a  short  presentation  of  the  robot  Nomad  and  the  sensors  we  use.  We 
will  then  present  the  solution  we  have  developed  to  improve  the  measurements  of  the 
ultrasonic  sensors.  The  tools  and  the  3D  application  will  then  be  presented. 


A  robot  called  ^^Nomad*^ 


General  description 

The  Nomad200  is  an  integrated  mobile  robot 
system  with  four  sensing  modules:  tactile, 
infrared,  ultrasonic  and  2D  laser  (Figure  1). 
The  Nomad200  has  an  on-board  multiprocessor 
system  consisting  of  a  80486  computer  and 
multiple  slave  microcontrollers.  This  system 
performs  sensor  and  motor  control,  host 
computer  communication,  and  supports  on¬ 
board  programming.  The  Nomad200  offers  an 
integrated  software  development  package  for 
the  host  computer  including  a  graphic  interface 
and  a  robot  simulator. 


Programming  theNomad200 


Figure  1:  The  NomadlOO 


The  Nomad200  can  be  programmed  and  controlled  in  two  ways:  directly  via  the  on-board 
PC  card  or  remotely  via  a  UNIX  workstation. 

In  the  first  case,  no  communications  are  possible  between  the  robot  and  the  remote  station 
and  the  user  has  no  idea  of  what  its  program  is  doing. 
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In  the  late  case,  commands  and  data  are  transmitted  by  an  Ethernet  radio  link.  The 
application  program  called  ‘client  program’  runs  on  the  workstation  and  sends 
commands,  via  the  host  interface  (Nserver),  to  a  communication  program  running  on  the 
robot  (Robot  Demon  or  robotd)^.  This  program  interprets  the  packets  sent  by  the  station 
and  sends  itself  the  data  of  the  sensors  to  the  workstation.  The  programmer  is  able  to  see 
not  only  what  its  program  is  doing  but  also  the  movements  of  the  robots  and  the  values  of 
the  sensors. 


Figure  2:  The  Nserver  GUI 
The  ultrasonic  sensors 

The  system  called  “Sensus  200”  is  a  ring  of  16  Polaroid  6500  sonar  ranging  modules.  The 
Polaroid  6500  is  an  acoustic  range  finding  device  that  has  been  widely  used  in  the  mobile 
robotics  community.  It  can  measures  distances  from  6  inches  to  35  feet,  with  a  typical 
absolute  accuracy  of  +/-  1  percent  over  the  entire  range. 

The  transducer  do  not  emits  energy  homogeneously  in  all  directions,  but  instead  form 
lobes  of  decreasing  intensity,  as  illustrated  in  figure  3. 

These  sensors  generate  a  lot  of 
uncertainties  in  the  measurements.  The 
main  sources  of  errors  are  the 
characteristics  and  the  position  of  the 
target.  Most  of  the  indoor  surfaces  act 
as  mirrors  with  respect  to  sound  waves. 
One  consequence  of  the  reflective 
properties  of  surfaces  is  the  multiple 
echo  effect:  the  sound  wave  bounces 
around,  and  eventually  reaches  the 
Figure  3:  A  typical  Emission  diagram  of  a  sonar  sensor  receiver  after  several  reflections. 


^  A  distributed  control  mode  have  been  developed  and  presented  in  [2] 
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A  good  representation  of  sonar  data  is  the  sonar 
scan:  a  dot  representing  the  measured  distance  is 
drawn  on  the  sonar  axis.  Repeated  measurements 
over  a  given  angular  range  give  a  sonar  scan. 
Sonar  scans  are  the  basic  data  that  can  be  used  for 
map  building  or  navigation  purposes. 

You  can  see  on  figure  4  a  typical  map  build  using 
sonar  data. 


The  mapping  of  the  environment 

In  order  to  obtain  a  usable  map  of  the  environment,  several  sensors  are  used  together  and 
successive  measurements  are  accumulated,  or  better  fused,  in  a  spatial  2D  map.  Several 
methods  exist  to  fused  sonar  data:  probabilistic  or  statistical  combination,  heuristics  rules, 
fuzzy  logic,  etc. 

The  idea  here  is  to  rejects  wrong  data  as  soon  as  possible,  with  other  words  before  the 
spatial  mapping.  Two  different  methods  have  been  developed:  the  first  one  uses  the  well- 
known  statistical  median  function  [3],  the  second  one  is  based  on  linear  regression  and 
the  estimation  of  future  readings  [4].  The  processing  is  made  for  each  sensor 
independently. 


The  time  sequence  median  filtering 

The  idea  is  to  remove  fast  measurements  variations  by  regarding  the  mean  evolution  of 
the  data.  This  processing  has  to  be  done  during  the  motion  of  the  robot  and  in  real-time. 

If  we  look  at  a  typical  time  sequence  recorded  during  a  motion  of  the  robot  we  see  that 
some  values  are  completely  different  from  their  neighbor  values.  The  median  function  is 
a  statistical  function  that  is  commonly  used  in  image  processing  to  smooth  the  image.  It 
acts  like  a  low-pass  filter  and  has  the  property  to  remove  some  noise  while  preserving 
edges. 

In  our  case  we  record  a  time  sequence  and  the  values  are  then  ordered.  The  measured 
value  is  then  replaced  by  the  middle  value  of  the  sequence.  You  can  see  several  examples 
here  below. 


Time  sequence 

Ordered  sequence 

12 

15 

15 

16 

12 

12 

12 

h 

15 

16 

15 

16 

15 

15 

14 

=> 

14 

15 

16 

35 

15 

false  data  rejected 

14 

15 

35 

35 

14 

15 

45 

35 

35 

15 

preservation  of 

15 

15  '35  35 

n 

Value  to  be  replaced 

36 

15 

15 

55  35 

. f'“ 

Middle  value 

35 

35 

n 

New  value 

edges 

In  a  spatial  filtering,  there  is  no  difficulties  to  consider  the  values  surrounding  the  center 
value  (the  value  to  which  we  applied  the  filter).  But  in  a  time  filtering  we  must  process 
the  data  recorded  before  and  after  the  value  we  want  to  replace.  The  consequence  is  that 
the  corrected  data  are  only  available  after  a  certain  delay.  The  duration  of  the  delay 
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depends  on  the  firing  rate  of  the  sensors  and  on  the  length  of  the  sequence  you  consider 
for  the  filtering. 

The  length  of  the  sequence  also  influences  the  size  of  the  wrong  data  we  can  reject.  If  we 
consider  a  sequence  of  N  elements,  we  can  eliminate  wrong  sequence  which  size  is  < 
(N+l)/2.  Otherwise,  the  value  will  be  replace  by  a  wrong  value. 

In  the  figures  5  we  can  see  the  results  of  a  processing  with  a  size  of  7  elements. 


donn^es  percues 


t1 


Figure  5a  and  5b:  original  and  processed  data 


The  best  results  were  generally  obtained  with  this  length.  In  this  case  we  can  reject  a 
sequence  of  3  successive  false  measurements  and  the  delay  introduced  is  not  to 
important. 

We  see  on  the  next  figure  a  complete  map  built  with  corrected  data. 
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In  order  to  suppress  the  delay  we  developed  the  method  explained  in  the  next  paragraph. 

The  time  sequence  filtering  by  linear  regression  and  estimation  of  future  readings 
With  this  processing  we  want  to  reject  the  wrong  data  generated  by  false  echoes.  The 
goal  is  to  correct  the  data  in  real  time  and  with  no  delay.  To  achieve  this  goal,  we 
consider  the  three  last  measurements  and  we  compute  the  straight  line  that  best  fits  these 
points  (Least  square  method).  We  can  now  compute  one  hypothesis  for  the  next  reading 
and  compare  with  the  real  value.  If  the  next  measurement  differs  too  much  from  the 
hypothesis,  we  cannot  just  drop  it;  several  cases  have  to  be  considered  because  this  new 
point  could  perhaps  own  to  a  new  obstacle.  So  we  have  to  compute  new  hypothesis  and 

compare  the  next  data 
with  these  hypothesis. 
We  repeat  this 

procedure  three  times 
and  if  the  last  point 
does  not  match  one 
hypothesis  we  stop  the 
processing  and  we 
restart  with  new  data. 
You  can  see  on  figure  7 
one  example  of 

corrected  data. 


before  processing  —W— after  processing  ] 


Figure  7:  Raw  and  corrected  data  for  one  sensor 
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We  can  also  compare  a  map  built  with  raw  and  corrected  data: 


Figure  8;  Without  correction 


Figure  9:  With  correction 


We  see  that  some  noise  remains.  These  isolated  points  can  now  be  eliminated  (if  wanted) 
by  a  simple  image  processing  (low  pass  filtering  or  something  like  that. . 

The  3D  visualization 
Motivation 

As  explained  in  the  introduction,  video  cameras  offer  a  poor  resistance  to  high  radiations 
level  and  an  altemati^'e  solution  is  needed  to  help  the  user  to  remotely  control  the  robot  or 
the  arm  for  manipulation,  A  3D  representation  of  the  environment  of  the  robot  is  certainly 
required.  The  data  acquired  by  the  ultrasonic  sensors  can  be  fused  with  existing  3D 
representation  of  the  installations  to  indicate  parts  and  obstacles. 

Other  requirements  for  this  application  were  to  use  low  cost  computer  to  perform  the 
visualization  and  to  get  rapidly  an  usable  system.  The  solution  we  propose  is  based  on 
today  Internet  tools:  VRML^  and  Java.  The  best  motivation  was  the  existence  of 
viewer/browser  and  the  portability  of  the  applications  written  in  Java. 


^  Similar  developments  are  made  at  the  kiss-lab  [5] 
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VRML 

The  Virtual  Reality  Modeling  Language  (VRML)  is  a  file  format  for  describing  3D 
interactive  worlds  and  objects^.  It  may  be  used  in  conjunction  with  the  World  Wide  Web. 
It  may  be  used  to  create  three-dimensional  representations  of  complex  scenes  such  as 
illustrations,  product  definition  and  virtual  reality  presentations. 

VRML  is  capable  of  representing  static  and  animated  objects  and  it  can  have  hyperlinks 
to  other  media  such  as  sound,  movies  and  images.  Interpreters  (browsers)  for  VRML  are 
widely  available  for  many  different  platforms  as  well  as  authoring  tools  for  the  creation 
of  VRML  files. 

VRML  is  a  new  standard  that  has  known  a  rapid  evolution  since  its  apparition  two  years 
ago.  VRMLl.O  provided  a  means  of  creating  and  viewing  3D  static  worlds;  VRML  2.0 
adds  interaction,  animation  and  extension  capabilities. 

You  can  find  viewers  for  all  platforms  and  operating  system  (OS)  but  each  of  them  runs 
on  a  limited  number  of  computers.  Some  viewers  can  run  as  standalone  application  and 
others  like  plug-in  in  your  favorite  browser.  At  present  day,  no  one  viewer  implements 
the  complete  VRML  specifications. 

In  VRML2.0  you  can  use  scripts  to  animate  objects  in  a  world  or  give  them  a  semblance 
of  intelligence.  This  script  can  be  written  in  any  programming  language  that  the  browser 
supports.  In  practice  it  is  Java  or  Javascript, 

JA  VA  -  Applets  and  Applications 

Java  is  an  object-oriented  programming  language  developed  by  Sun  Microsystems.  The 
main  purposes  of  the  developers  were  to  create  a  language  that  let  programmers  write 
programs  that  can  be  executed  on  every  computer  whatever  its  processor  or  OS. 

Sun  introduced  the  concept  of  Java  Virtual  Machine.  When  you  compile  a  program 
written  in  Java,  you  produce  bytecode  that  will  be  interpreted  by  the  Java  interpreter 
during  its  execution.  Another  goal  was  to  create  a  language  that  will  permit  to  write 
distributed  and  network  applications  more  easily.  Java  can  be  used  to  write  serious  big 
applications  (see  Corel  WordPerfect  suite  for  Java)  or  simple  applet  to  enhance  WEB 
pages.  Applets  are  Java  programs  that  are  embedded  in  Web  pages.  The  code  is 
downloaded  with  the  page  and  executed  on  the  computer  of  the  netsurfer.  Of  course  to  do 
this  you  must  have  a  compatible  Java  browser. 

Javascript  is  totally  different  from  Java.  It  is  a  scripting  language  that  is  interpreted  by  the 
browser  and  was  introduced  by  Netscape;  conunands  are  embedded  in  HTML  pages  as 
text. 

In  the  following  table  you  can  see  a  sununary  of  the  principal  viewer/plug-ins  for  VRML 
that  will  run  with  Linux 


Name 

Works  as: 

for: 

on.  platform: 

LiquidReality 

ffrrrWnn^Wi 

Win95 

LinuxAVin95/Sun/SGI 

DimensionX 

Vrwave  (beta) 

Application 

JDK 

Linux/SGI/Sun 

ncM 

^  See  the  complete  specifications  at  http.7/www.vrml.orgArRML97/DIS 
^  For  a  complete  review  see:  http://www.sdsc.edii/vrml 
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We  opted  for  the  LiquidReality  (Lr)  package  because  of  existing  versions  for  Linux  and 
Windows95  and  the  possibility  to  write  standalone  applications. 

Liquid  Reality  is  a  set  of  Java  classes  that  implement  the  VRML2.0  specifications.  It  adds 
the  capabilities  of  3D  technology  to  the  power  of  Java.  It  provides  a  rich  set  of  function 
calls  from  Java  that  allow  the  developers  to  create  and  manipulate  simple  or  complex  3D 
scenes  as  an  integral  part  of  their  applications. 

With  the  3D  programming  interface  for  Java,  the  developer  has  the  ability  to  create 
applications  that  can  read  in  and  allow  the  user  to  view  and  interact  with  VRML2.0 
worlds.  In  addition,  the  developer  has  a  new  level  of  control  over  these  worlds.  Nodes 
can  be  crated,  deleted  and  modified  at  mntime.  Complex  behaviors  can  be  triggered  when 
VRML  or  standard  window  events  are  generated. 

The  visualization  application:  3DApp 

The  main  functionality  of  our  application  is  to  visualize  the  motion  of  our  robot  in  3D. 
This  program  conununicates  with  the  Nserver  program  (see  $2.2)  and  continuously  reads 
the  coordinates  of  the  robot  and  its  orientation;  the  drawing  is  updated  at  each  iteration. 
Normally,  to  command  the  robot  via  the  Nserver,  we  have  to  write  a  C  program.  The  Java 
language  offers  the  possibility  to  call  native  functions,  that  is,  functions  written  in  an 
other  language  (C,C++,...  ).  So  the  developer  can  use  optimized  rendering  functions  or 
existing  specialized  API  (loosing  the  portability). 

The  solution  we  adopted  was  to  implement  the  function  that  permits  the  communication 
with  the  Nserver  as  a  library  and  to  call  it  from  a  Java  program. 

Java  offers  different  solution  for  the  implementation:  the  3DApp  can  be  executed  as  a 
stand-alone  application  or  as  an  applet  in  a  browser  (or  in  Appletviewer).  Due  to  security 
and  access  permissions,  an  application  is  easier  to  write. 

An  usual  client  program  for  the  Nserver  communicates  with  it  via  sockets,  consequently, 
it  can  run  on  the  same  or  on  a  different  computer  (but  with  Linux)  than  the  Nserver.  So 
the  3Dapp  application  can  run  on  a  remote  computer  (if  this  computer  has  a  Java  Virtual 
Machine). 


LINUX 

iNservcr 


UNIXAJNUX 


3DApp 
Java  +  C 


Figure  10:  Simple  communication:  the  2  applications 
can  run  on  the  same  or  on  different  machines. 

If  we  want  to  let  our  program  run  on  different  computers  and  different  operating  systems, 
we  can  implement  one  of  these  solutions: 

On  the  Linux  server  machine  we  can  write  a  mix  Java/C  program  that  wiU  act  as  a  server 
for  the  3DApp  program  or  3D  Applet  applet. 
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LINUX 

Nserver 

UNIX/LINUX 


Server 
Java  +  C 


LINUX/UNIX/WIN95 

Browser  +  Plug-in 

3DApplet 

Figure  11;  Communication  via  a  server 


With  this  configuration  we  can  chose  an  other  viewer/plug-in,  if  we  want  for  example 
make  the  visualization  on  a  MS-Windows  computer.  In  this  case  the  speed  of  the  drawing 
could  be  improve  using  a  JIT^  compiler  (not  yet  available  for  Linux) 

The  3Dapp  can  also  be  used  as  a  standalone  application  (not  connected  to  the  Nserver). 
The  user  has  the  possibility  to  drive  the  robot  with  a  virtual  joystick.  You  can  see  a  screen 
capture  of  the  interface  in  the  figure  12. 


Future  developments 

The  developments  we  have  planned  are  the  following: 


•  the  implementation  of  the  second  communication  solution  (see  above), 

•  the  command  of  the  real  robot  via  the  3D  GUI, 


^  A  Just-in-Time  compiler  compiles  the  Java  bytecode  into  machine  code. 
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•  The  VRML  standard  does  not  handle  collisions  between  objects  but  only  between  the 
viewer  (called  avatar)  and  objects.  In  order  to  use  this  application  as  simulator  this 
behavior  should  be  added  to  the  objects. 

•  At  this  stage,  the  user  can  navigate  freely  into  the  scene.  An  interesting  option  should 
be  the  automatic  following  of  objects  and  the  automatic  selection  of  the  best 
viewpoint. 


Conclusions 

Today  we  see  very  sophisticated  user-interfaces  with  multi  cameras,  HUD,  etc...  In 
nuclear  applications  quite  simple  sensors  are  used  because  they  resist  well  to  radiations 
doses.  This  work  addresses  two  problems  introduced  by  this  restriction:  the  improvement 
of  ultrasonic  data  by  time  filtering  and  the  representation  of  the  3D  environment  of  the 
robot. 

The  developed  algorithms  for  the  data  filtering  have  proved  to  be  robust  and  improve  the 
quality  of  returned  data. 

The  3D  representation  is  at  the  beginning  of  its  development.  It  is  based  on  today  Internet 
standards  and  runs  on  several  platforms. 
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Abstract 

A  six-axis  hand  controller  designed  to  use  the  high  sensitivity  of  the  human  hand-fmger- 
system  is  presented.  The  applied  input  force  is  measured  by  a  force/torque-sensor  and 
proportionally  transformed  to  control  the  motion  and  the  output  force  of  a  six-axis  in¬ 
dustrial  robot.  The  motor  feedback  of  the  robot  position  to  the  hand  controller  gives  the 
operator  the  feeling  of  handling  the  robot’s  load  with  his  own  hand.  By  means  of  a  sim¬ 
ple  task  which  is  matching  a  cylindrical  peg  into  nine  holes  with  different  diameters  and 
chamfers  the  operator’s  performance  is  evaluated.  The  operator’s  forces  and  the  peg  po¬ 
sitions  are  recorded  while  the  task  is  carried  out.  The  time  required  to  complete  the  task 
(sequential  match  into  each  one  of  the  holes)  is  the  main  indicator  of  the  operator’s  per¬ 
formance. 

Research  has  been  made  on  the  influence  of  some  parameters  like  force  /  torque  resolu¬ 
tion  (by  time  and  value)  on  force  measurement  as  well  as  translation  and  rotation  scales 
on  the  position  /  orientation  feedback.  Results  show  best  performance  on  scales  that  cor¬ 
respond  to  the  finger  workspace  if  the  hand  is  supported.  For  the  examined  task,  the  ori¬ 
entation  feedback  comes  out  to  be  more  important  than  the  position  feedback. 

The  force  /  torque  transmission  parameters  might  not  be  necessarily  as  precise  as  the 
human  fingers  performing  the  same  task  under  optimum  circumstances  since  the  dy¬ 
namics  of  the  controlled  manipulator  is  more  restricting.  Complementary  inquiries  have 
been  made  on  knob  size  and  shapes. 

Introduction 

The  optic,  acoustic,  kinesthetic  and  haptic  senses  have  evolved  to  constitute  the  most 
important  information  channels  for  people  to  perceive  the  environment.  On  controlling 
machines,  some  information  can  be  perceived  without  technical  displays  (e.g.  accelera¬ 
tion,  noise,  track  while  driving  vehicles),  others  must  be  provided  by  those  displays  (e.g. 
velocity  and  height  in  airplanes).  Conventional  displays  mostly  use  the  optical  and 
sometimes  the  acoustical  channel  (e.g.  for  speech  output  and  alert  signals). 

The  haptic  channel  is  essentially  used  for  recognition  of  forms  and  surfaces  as  well  as  to 
determine  the  force  which  is  applied  by  or  to  the  surroundings.  Tasks  like  inserting 
screws  into  threads,  cutting  soft  things  on  hard  surfaces,  or  digging  in  non-homogenous 
soils  are  much  harder  to  complete  without  the  haptic  information  (=  force  and  position 
information). 


When  controlling  machines  hke  cranes  and  power  shovels,  this  haptic  sense  usually  gives 
feedback  on  the  information  transmitted  to  the  machine,  but  not  about  its  internal  state 
(e.g.  position  and  forces).  For  effective  appHcation  of  the  highly  sophisticated  haptic 
sense,  whose  reaction  time  and  perception  thresholds  are  significantly  lower  compared  to 
the  other  senses,  these  machine  states  are  displayed  on  active  hand  controllers.  Thus,  the 
operator  handles  his  tool  intuitively  like  if  the  tool  was  in  his  own  hands.  By  the  use  of 
highly  trained  habituation  patterns,  learning  time  and  mental  work  load  are  reduced,  and 
at  the  same  time  dangerous  situations  can  be  overcome  safer  and  faster. 

Regarding  the  high  motorial  precision  and  sensitivity  of  the  hand-finger-system  for  the 
control  of  many  kinds  of  manipulators,  near  or  distant,  the  Active  Hand  Controller  „Spi- 
der“  (Fig.  1)  has  been  developed,  constructed  and  tested  at  the  Institute  of  Ergonomics. 
In  comparison  to  typical  manual  input  devices  (joysticks,  wheels,  pedals)  it  can  provide 
full  haptical  feedback.  This  is  achieved  by  a  force/torque-sensor  which  measures  the  ap¬ 
plied  force  and  a  positioning  unit  which  displays  positions  and  orientations.  It  is  de¬ 
signed  as  a  6-degrees-of-freedom  device  to  be  a  universal  platform  for  experiments  and 
applications. 


Figure  1:  Active  Hand  Controller  „SpideC 

The  advantages  of  force  and  position  display  in  comparison  to  conventional  controllers 
are  well  known.  The  aim  is  now  to  quantify  the  influence  of  single  system  parameters  on 
human  performance  in  bilateral  teleoperation  with  haptic  feedback.  In  this  way,  the  ex¬ 
tensive  control  and  hardware  needs  can  be  reduced  to  a  cost  effective  minimum  without 
decrement  of  human  control  capabilities. 


System  Description 

The  used  teleoperation  setup  consists  of  the  Active  Hand  Controller  „Spider“  as  the 
Master  in  combination  with  a  Puma  560  industrial  robot  as  the  Slave.  The  setup  involves 
principally  two  independent  control  chains: 

•  Proportional  transfer  of  hand-/finger  forces  to  the  manipulator  (Slave). 

•  Position  /  velocity  feedback  of  the  manipulator's  action  to  the  fingers  of  the 
operator. 

The  environment  of  the  manipulator  and  the  operator  close  the  control  chains  to  a  control 
loop  (Fig.  2). 
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Figure  2:  Control  scheme  of  the  teleoperation  system 

Transmission  of  Forces  from  the  Hand  Controller  to  the  Manipulator: 

Forces  and  torques  of  the  hand  and  fingers  are  measured  by  a  6-axis  force/torque-sensor. 
They  have  to  be  output  by  the  manipulator  in  two  different  manners: 

•  as  a  velocity  (without  environmental  contact) 

•  as  a  force  (in  constrained  motion) 

The  key  point  in  the  transmission  control  is  to  choose  between  velocity  and  force  output. 
Therefore,  the  default  velocity  control  is  replaced  by  force  control  if  tlie  motor  currents  of 
the  robot  (which  are  proportional  to  the  motor  torque)  exceed  the  value  that  the  input 
force  allows.  The  application  of  these  two  schemes  provides  controlled  movement 
through  the  free  space  as  well  as  relatively  sensitive  handling  when  in  contact  with  ob¬ 
jects. 
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Position  Feedback  from  the  Manipulator  to  the  Hand  Controller 

The  position  of  the  robot ^s  tool  center  point  is  calculated  from  the  built-in  angle  encod¬ 
ers.  TTie  joint  positions  are  transformed  separately  by  translation  and  rotation  into  Carte¬ 
sian  coordinates,  which  can  be  displayed  proportionally  by  the  hand  controller. 

Hand  Controller  Mechanics 

The  positioning  unit  of  the  hand  controller  is  a  complete  robot  with  6  degrees  of  freedom. 
The  essential  difference  to  typical  industrial  robots  is  the  kinematics  which  is  chosen 
following  the  mobility  and  sensitivity  of  the  human  hand-finger-system.  It  consists  of 
linear  units  with  universal  joints  at  both  sides,  connecting  a  base  plateau  with  a  head 
plateau  (Fig.  1).  Special  care  has  been  taken  to  construct  joints  without  play  and  wide 
angular  limits  at  the  head  plateau. 

The  construction  is  based  on  a  geometry  optimized  and  checked  by  computer  calcula¬ 
tions.  A  postulation  was  a  rotatory  mobility  of  30  degrees  around  any  axis.  Thus,  the 
maximum  workspace  of  the  positioning  unit  is  2,3*10^  mm^  which  contents  a  symmetri¬ 
cal  workspace  of  1,3*10^  mm^  (Fig.  3).  This  workspace  is  greater  than  the  area  a  sup¬ 
ported  hand  can  reach. 


Figure  3:  Maximum  symmetrical  workspace  of  the  ^Spider"'  (Mobility  30°  around  any 

axis) 


The  actuators  are  stepping  motors.  Damping  elements  suppress  excessive  noise  and  high 
frequency  oscillations.  The  position  accuracy  is  strongly  depending  from  the  actual  posi¬ 
tion,  but  always  better  than  10  pm.  Horizontal  maximum  speed  is  125  mm/s,  maximum 
acceleration  and  deceleration  values  are  5.1  m/s^  and  41  m/s^.  Vertical  rates  are  about  the 
half  of  this  figures.  This  high  deceleration  makes  even  hard  impacts  well  recognizable 
haptically. 

Experimental  Setup 

[Hannaford  et  al  (1991)]  presented  a  frequently  cited  experiment  for  teleoperation.  The 
task  is  to  insert  a  peg  (diameter  25.35mm)  into  9  cylindrical  holes.  The  holes  are  placed 
in  a  3x3  matrix  and  differ  by  diameter  (25.4lmm  to  25.47mm)  and  chamfer  (0.4mm*45° 
to  1.6mm*45°).  The  task  has  a  Fitts  difficulty  from  7.8  to  9.  The  time  needed  to  insert 
the  peg  sequentially  into  the  holes  (twice  for  each  pass)  is  an  indicator  for  the  operator's 
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performance.  It  also  depends  on  the  system's  parameters  and  indicates  the  ergonomic 
aptitude  of  the  parameters  set. 

For  the  experiments,  an  arm  and  hand  support  which  is  free  movable  horizontally  is  pro¬ 
vided.  The  subjects  have  to  train  the  task  a  certain  number  of  times  so  that  learning  ef¬ 
fects  disappear.  The  holes  to  be  inserted  are  indicated  by  LEDs.  The  task  can  not  proceed 
if  the  desired  insertion  is  not  complete,  so  that  missing  or  incomplete  insertions  never 
occur.  The  time  needed  to  perform  the  whole  cycle  is  reduced  to  a  fit-in  time,  which  is 
the  remainder  when  the  time  from  leaving  the  last  hole  to  approaching  (1cm)  the  next 
one  is  eliminated.  This  reduction  (amounts  an  average  of  only  18%  of  the  complete  pass) 
is  made  to  ignore  unconstrained  movements  and  single  mistakes  in  the  hole  order  made 
by  the  subjects. 

Experiment  1:  Scales 

In  this  experiment,  the  influence  of  the  scale  in  which  robot  movements  are  transformed 
into  movements  of  the  hand  controller  are  examined.  It  can  be  distinguished  between  the 
scales  of  position  and  orientation.  Four  different  scale  combinations  are  tested: 

•  A:  position  1:1,  orientation  1 : 1 

•  B:  position  3:1,  orientation  3: 1 

•  C:  position  6:1  orientation  1:1 

•  D:  position  16:1,  orientation  16:1 

Hereby,  at  combination  A  the  workspace  of  the  hand  controller  is  widely  utilized,  while 
D  represents  almost  an  isometric  controller  with  very  small  movements. 

Results  of  Experiment  1 : 

There  are  50  subjects  with  an  average  age  of  27  years  (a=6.4)  performing  the  task.  Af¬ 
terwards  they  are  presented  a  questionnaire  to  judge  the  different  scale  combinations. 

Fit-in  times:  The  following  times  are  achieved  at  the  four  combinations: 


Scale  combination 

Fit-in  time  t  [s] 

Std.-Dev.  a  [s] 

A(Pos.  1:1,  Or.  1:1) 

102.1 

26.9 

B(Pos.  3:l,Or.3:l) 

101.4 

25.9 

C(Pos.6:l,Or.  1:1) 

75.0 

19.0 

D(Pos.l6:l  Or.  16:1) 

113,2 

32.3 

Table  2:  Fit-in  times  for  experiment  1 

The  null  hypothesis  „A  does  not  differ  from  B“  must  be  accepted  by  a  probability  of 
0.75,  All  other  combinations  are  different  by  a  very  high  degree  of  statistical  signifi¬ 
cance. 
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Questionnaire: 

The  judgement  on  the  order  of  preference  constitutes  the  most  relevant  item  in  the  ques¬ 
tionnaire.  38  of  50  subjects  vote  for  combination  C  as  the  most  favorable  one,  26  judge 
D  as  the  least  preferable  one.  The  question  about  the  necessary  concentration  at  each  one 
of  the  combinations  show  up  similar  results.  Most  concentration  is  needed  at  D,  least  at 
C.  In  both  cases,  the  combinations  A  and  B  receive  average  judgements  without  signifi¬ 
cant  differences. 

Experiment  2:  Force  Parameters 

This  experiment  covers  some  parameters  of  the  force  /  torque  measurement  at  the  hand 
controller.  As  the  implemented  sensor  is  able  to  work  at  high  sampling  rates  (up  to  520 
Hz)  and  a  resolution  of  10  bit,  the  point  of  interest  is  if  lower  precision  would  reduce  the 
system's  overall  performance.  As  it  has  been  shown  by  some  physiological  experiments, 
the  human  fingers'  sensitivity  can  beat  the  above  benchmarks  under  ideal  circumstances. 
The  following  four  force  parameters  are  chosen  for  this  experiment: 

•  E:  (Ideal)  Sampling  rate  520  Hz,  Force  resolution  0.02N,  Torque  resolution 
0.0002Nm 

•  F:  Sampling  rate  65  Hz,  Force  resolution  0.02N,  Torque  resolution  0.0002Nm 

•  G:  Sampling  rate  22  Hz,  Force  resolution  0.02N,  Torque  resolution  0.0002Nm 

•  H:  Sampling  rate  520  Hz,  Force  resolution  IN,  Torque  resolution  0.  INm 
The  position  scale  is  set  according  to  the  result  of  the  previous  experiment  (6:1  for  posi¬ 
tion,  1:1  for  orientation).  That  is  why  the  maximum  speed  of  the  manipulator  is  raised  in 
this  experiment  in  order  to  adjust  it  to  the  speed  range  of  the  hand  controller. 

Results  of  Experiment  2: 

There  are  24  subjects  with  an  average  age  of  32  years  (a=10.5)  performing  the  task.  Af¬ 
ter  the  experiment,  a  similar  questionnaire  to  experiment  1  has  to  be  completed. 

Fit-in  times: 

The  subjects  achieve  the  following  fit-in  times  with  the  different  force  parameters: 


Force  Parameter 

Fit-in  time  t  [s] 

Std.-Dev.  a  [s] 

E  (Ideal) 

58.6 

19.7 

F  (Sampling  Rate  65  Hz) 

61.5 

23.0 

G  (Sampling  rate  22  Hz) 

75.5 

30,1 

H  (Force/Torque  resolution  1  N  / 
0.1  Nm) 

60.8 

21.3 

Table  2:  Fit-in  times  for  experiment  2 


The  null  hypothesis  must  be  accepted  between  the  parameter  sets  E  and  H  as  well  as  F 
and  H,  and  there  is  a  very  high  significant  difference  between  G  and  the  other  three  pa¬ 
rameter  sets. 

Questionnaire: 

According  to  the  objective  data,  subjects  are  able  to  judge  G  significantly  as  the  worst 
parameter  set,  but  can  not  state  differences  between  the  other  ones. 

Knob  Judgement 

During  the  two  experiments  mentioned  above  an  accompanying  inquiry  concerning  knob 
characteristic  is  made.  The  two  utilized  knobs  are: 

•  A  43mm  plastic  ball  with  regular  hollows  (golf  ball)  in  experiment  1 

•  A  60  nun  wooden  bowl  in  experiment  2 

Questions  are  asked  about  the  size  and  the  surface  of  the  knob.  The  judging  criteiia  is  a 
scale  with  5  items  ranging  from  too  small  to  too  big  and  too  smooth  to  too  rough,  re¬ 
spectively. 


The  results  are  as  follows: 


Experiment  1 
(SOSubj.) 

Size 

too  small 
30% 

a  bit  too  small 
40% 

right 

24% 

a  bit  to  big 

4% 

too  big 

2% 

43mm  diame¬ 
ter 

Surface 

too  smooth 
10% 

a  bit  to  smooth 
36% 

right 

52% 

a  bit  too  rough 
2% 

too  rough 

0% 

Experiment  2 
(24Subj.) 

Size 

too  small 
0% 

a  bit  too  small 
25% 

right 

41.7% 

a  bit  to  big 
33.3% 

too  big 

0% 

60mm  diame¬ 
ter 

„Wooden 

Bowl“ 

Surface 

too  smooth 
16.7% 

a  bit  to  .smooth 
37.5% 

right 

41.7% 

a  bit  too  rough 
4.2% 

too  rough 

0% 

Table  3:  Judgements  on  knob  sizes  and  surfaces 


Regarding  the  form  of  the  knob,  there  are  two  opposite  judgements:  about  the  half  of  the 
subjects  would  prefer  some  kind  of  cylindrical  knob  shape,  the  other  half  does  not.  Inde¬ 
pendently,  most  subjects  would  prefer  some  bigger  grip-holes  and  a  more  anatomic  shape 
to  augment  the  perception  of  orientation. 

Conclusions 

In  Experiment  1,  scale  combination  C,  the  only  one  with  scales  different  by  position  and 
orientation  (6:1  resp.  1:1),  is  evaluated  subjectively  as  well  as  objectively  as  the  best 
combination.  Following  factors  contribute  to  achieve  this  result: 

1)  Orientation  feedback  is  more  important  for  the  performed  task  than  position  feedback, 
that  is  why  combination  B  (3: 1,3:1)  and  combination  D  (16:1,16:1)  are  less  advanta¬ 
geous. 

344 


2)  The  workspace  of  the  human  hand-fmger-system  is  well  utilized.  The  arm  support  can 
be  put  under  the  ball  of  the  hand  so  that  full  advantage  can  be  taken  of  the  finger  sen¬ 
sitivity  (not  so  in  combination  A(l:l,l*l))- 

3)  High  velocities  at  the  hand  controller,  especially  translatory  ones  (like  at  A),  interfere 
with  the  haptical  feedback  by  out-of-round  action  of  some  linear  units.  These  vibra¬ 
tions  of  about  15  Hz  are  transferred  to  the  entire  hand  controUer.  Furthermore,  the 
motion  of  the  six  units  is  not  coordinated  but  each  one  is  heading  for  its  desired  posi¬ 
tion  as  fast  as  possible.  Thus,  a  maximum  straightness  of  the  movements  is  not  guar¬ 
anteed. 

On  comparing  the  scale  combinations  A,  B  and  D  the  following  conclusions  can  be 
drawn: 


•  The  factor  1)  balances  the  factors  2)  and  3)  comparing  A  with  B 

•  The  almost  isometric  combination  D  provides  too  less  haptic  information 
so  that  the  superiority  of  hand  controllers  that  reflect  force  and  position 
has  once  more  been  proven. 

Experiment  2  shows  clearly  that  the  force/torque  parameters  may  not  meet  the  require¬ 
ments  of  the  human  sensitivity  under  optimum  circumstances.  The  dynamics  of  the  con¬ 
trolled  manipulator  and  the  control  loop  is  supposed  to  be  the  benchmark  for  force  pa¬ 
rameters.  In  every  case,  a  stable  control  of  the  system  implies  lowpassing  of  the  input 
force  so  that  even  rough  quantizations  by  time  (65  Hz,  „F‘)  and  value  (1  N  ,  0.1  Nm, 
„H“)  do  not  impair  human  overall  performance.  Only  a  very  slow  sampling  rate  (22  Hz, 
„G“)  has  a  perceivable  damping  effect  on  manipulator  control  so  that  a  difference  in  fit- 
in  times  and  subjective  rating  can  be  measured. 

Regarding  the  two  experiments,  some  recormnendations  on  the  design  of  an  active  hand 
controller  for  teleoperation  purposes  with  force  input  and  position  feedback  can  be  for¬ 
mulated: 

•  The  scale  for  rotation  and  translation  should  be  chosen  to  fill  the  hand-finger  work¬ 
space  so  that  the  hand  can  be  supported.  The  separation  of  position  and  orientation 
scales  is  not  problematic, 

•  The  resolution  (by  time  and  value)  of  the  force/torque  sensor  does  not  have  to  meet 
highest  requirements.  It  can  be  oriented  by  the  system ^s  overall  dynamics, 

•  A  knob  size  of  60mm  (bowl)  was  judged  convenient  in  average  by  all  subjects.  Some 
grip-hole  like  structures  are  desirable. 

Further  research  is  planned  to  be  made  on  other  force  parameters  (offset  &  hysteresis) 
and  minimum  requirements  to  the  hand  controller's  mechanical  components  to  be  func¬ 
tional  without  loss  of  human  performance. 
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Abstract 

This  paper  is  based  on  two  case  studies  which  gave  the  authors  a  great  deal  of  insight  into 
the  development  of  Virtual  Reality  (VR)  systems.  The  authors  have  chosen  to 
diiferentiate  two  types  of  observations:  those  related  to  development  process,  and  those 
related  to  design  issues. 

RECMUVI  and  TILE  VIZ  are  VR  systems.  The  first  focuses  on  the  environmental 
education  of  10-12  year  olds  on  urban  solid  waste  recycling.  The  second  focuses  on  the 
selection  and  3D  visualization  of  floor  and  wall  ceramic  tiles. 

In  both  projects  two  different  usability  techniques  were  employed:  heuristic  evaluations, 
made  by  the  authors,  and  participative  evaluations,  made  by  end-users.  Each  technique 
raised  different  (but  sometimes  overlapping)  usability  problems.  The  use  of  the  ‘thinking- 
aloud’  technique  was  questioned,  as  the  possible  demands  on  cognitive  resources  affect 
subject’s  explorations  of  the  virtual  world  (RECMUVI).  Group  interviews  with  end-users 
focusing  on  a  checklist  of  usability  evaluation  criteria  themes  were  made.  These  meetings 
proved  useful  because  they  enable  end-users  to  report  their  experiences  and  discuss  the 
central  aspects  of  the  interactions. 

The  main  design  problems  were  related  to  the  need  to  generate  virtual  environments  with 
cues  that  orientate  the  navigation  and  representation  of  the  virtual  space.  The  need  to 
develop  navigation  systems  that  do  not  present  cognitive  overhead  and  the  possibility  of 
the  co-existence  of  different  navigation  types,  each  one  with  different  motor  and 
cognitive  requirements,  was  also  raised. 

Introduction 

Human  computer  interaction  evaluation  is  common  practice  in  computer  systems  design. 
Through  the  use  of  evaluation  tests  designers  want  to  assure  systems  adaptation  to  end- 
users.  And  this  is  true  not  only  of  ordinary  computer  systems  but  also  of  more  innovative 
technology  (Virtual  Reality  -  VR  -  is  only  an  example).  These  evaluations  gain  a 
growing  importance  when  designers  want  to  generalize  innovative  technology  for  non¬ 
expert  users. 

VR  systems  have  special  features  very  different  from  those  of  other  computer  systems, 
demanding  complex  interaction  styles.  First  of  all,  one  of  the  main  goals  of  VR  systems  is 
the  transmission  to  users  of  a  feeling  of  presence  in  a  computer-generated  (virtual)  world. 


'  The  authors  wish  to  thanks  to  Prof.  Antdnio  Sousa  Camara,  Head  of  Environmental  Systems  Analysis  Group  (GASA) 
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That  feeling  of  immersion  is  the  computer  generated  illusion  of  the  user  being  inside  a 
virtual  environment.  According  to  Hayward  (1993),  the  brain  is  the  last  artifice  of 
immersion.  Furthermore,  Varela  (1995)  argued  that  VR  can  be  thought  of  as  a  process  of 
excess  of  signification  and  the  ability  of  the  mind  as  a  neuronal  “narratives”  generator. 
Pimentel  and  Teixeira  (1993)  consider  "'The  question  isn't  whether  the  created  virtual 
world  is  as  real  as  the  physical  world,  but  whether  the  created  world  is  real  enough  for 
you  to  suspend  your  disbelief  for  a  period  of  time.''  (p.  15),  a  concept  shared  by  Slater  and 
Usoh  (1993).  These  last  authors  stated  that  the  human  partner  builds  the  world  through 
those  displays  and  acts  in  an  extended  virtual  space  generated  through  the  interaction 
between  perceptual  system  and  computer-generated  displays. 

Second,  and  as  a  way  to  assure  the  feeling  of  presence,  VR  systems  generally  have 
complex  user  interfaces.  Immersion,  or  the  feeling  of  presence,  in  a  virtual  world  almost 
always  requires  a  visualization  tool  (e.g.,  a  Head  Mounted  Display  -  HMD),  coupled 
with  a  navigation  and/or  object  manipulation  tool. 

Therefore,  the  need  to  better  understand  VR  systems  characteristics  and  their  cognitive 
demands  are  essencial  to  design  better  VR  systems,  i.e.  systems  adapted  to  end-users.  In 
this  paper,  we  present  two  case-studies  we  followed  during  the  past  two  years,  where  VR 
systems  characteristics  were  analysed. 

RECMUVI  background 

RECMUVI  was  a  project  to  develop  a  VR  game  applied  to  environmental  education  on 
urban  solid  waste  recycling,  aiming  to  generate  and  develop  skills  in  10-12  years  old  on 
collecting  and  selection  of  recyclable  materials  (Caldeira,  Dias,  Otero,  &  Silva,  1995). 
The  evaluation  of  the  interface  was  made  for  die  indoors  situation,  were  the  players 
should  collect  recyclable  objects  and  put  them  in  the  respective  boxes. 

Tools.  This  game  was  developped  in  a  Pentium  90,  using  WorldToolKit  of  SenseS.  Due 
to  technical  restrictions  the  house  was  broken  down  into  six  models  (each  model 
representing  a  room  -  corridor  and  five  other  rooms). 


The  game  used  a  HMD  (Virtual  Research)  with  a  visual  field  of  90  degrees  horizontal 
and  35  degrees  vertical,  a  navigation  tool  (Trigger,  Figure  1)  with  a  button  for  object 
manipulation,  and  one  (or  two,  depending  on  the  prototype)  Polhemus  track  sensors. 


Button 

Trigger 


Figure  1.  Trigger 


Prototypes.  Prototype  1  representes  three  rooms  in  the  virtual  home:  the  kids’  room, 
bathroom  and  corridor.  The  viewpoint  is  given  through  head  movements.  Navigation  was 
controlled  by  two  tools:  the  trigger  control  the  navigation  velocity  (only  forward 
movements),  and  the  HMD  control  the  navigation  direction. 

Object  manipulation  included  a  visual  cue  -  a  virtual  hand  -  and  two  different  actions: 
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approaching  the  object  and  touching  it,  or  the  surrounding  area,  with  the  virtual  hand. 
This  last  action  makes  a  yellow  wire  frame  appear  around  the  object.  Only  then  should 
the  user  press  the  trigger  button  selecting  the  object  (which  became  attached  to  the  virtual 
hand).  In  order  to  drop  the  object,  the  user  must  press  the  same  button  a  second  time. 

The  corridor  was  the  only  model  connected  to  the  other  models,  the  entrance  and  exit  of 
rooms  was  made  through  a  portal,  with  a  waiting  time  filled  with  a  fixed  white  image. 

Prototype  2  includes  all  rooms:  parents’  room,  kids’  room,  dining  room,  kitchen, 
bathroom,  and  corridor.  This  prototype  differs  fi'om  the  first  one  in  the  following  aspects: 
introduction  of  lights  that  simulate  shadows  and  contrasts;  different  height-width  ratio; 
introduction  of  various  object  and  boxes  in  blue,  green,  yellow,  and  red  colours;  the  white 
in  portals  changed  into  a  less  aggressive  blue. 

In  prototype  3  the  corridor  doors  that  where  closed  on  the  first  two  prototypes  are  now 
opened,  with  the  visualization  of  each  room  interior. 

The  navigation  scheme  changed.  The  navigation  direction  is  controlled  by  the  trigger. 
Hand  movement  corresponds  to  changes  in  the  visualization  of  an  arrow  -  that  replaced 
the  virtual  hand.  This  functionality  was  introduced  with  the  help  of  another  Polhemus 
sensor  (one  in  the  HMD  -  to  the  viewpoint  -  and  the  other  one  on  the  top  of  the  trigger  - 
to  the  navigation  direction), 

TILE  VIZ  background 

TILE  VIZ  is  the  code  name  adopted  for  Virtual  Decor™  1.0^  during  the  development 
process  and  is  the  name  used  throughout  this  paper,  TILE  VIZ  is  a  tool  to  help  users  in 
selecting  and  visualizing  ceramic  tile  patterns.  Therefore,  TILE  VIZ  is  a  design 
application  of  WC’s  and  Kitchens  where  users  can  place  typical  objects  and  select  and 
visualize  these  objects  and  wall/floor  ceramic  tiles. 

The  evolution  of  the  project  demanded  the  development  of  three  interdependent  modules. 
In  the  first  one  users  can  design  the  WC  and/or  the  kitchen  and  place  various  typical 
objects  in  the  designed  room/s  (doors,  windows,  lights,  bathtub. . .). 

In  the  second  module  users  can  select  tile  patterns.  They  can  visualize  the  design  rooms 
and  selected  objects  and  tiles  in  the  last  module,  the  real  VR  model.  In  this  model  users 
can  navigate  and  do  various  tile  manipulations. 

Tools,  TILE  VIZ  was  developped  in  a  Pentium  120,  using  a  toolkit  from  Microsoft.  The 
visualization  tool  (similar  to  die  one  used  on  RECMUVI)  was  a  HMD  from  Virtual  i-O. 

In  the  first  prototype,  the  navigation  (with  6  degrees  of  freedom)  was  controled  by  two 
tools:  the  HMD  controling  the  navigation  direction;  and  the  trigger  controling  the 
navigation  velocity. 

In  the  second  prototype  the  trigger  button  is  active  and  users  can  do  several  tile 
manipulations  (e.g.  change  of  patterns). 
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Virtual  Decor™  1.0  is  a  trademark  of  Soexporta,  Lda. 
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In  the  third  prototype  the  trigger  was  replaced  by  a  pad  (Figure  2),  with  an  extended  set 
of  functionalities:  users  can  not  only  change  tile  patterns  but  also  move  bars  and  wall  tiles 
upwards  and  downwards. 


Prototypes.  The  first  prototype  (very  priihitive),  was  only  a  part  of  the  final  VR  model: 
one  room  with  a  door,  a  panoramic  window  and  four  bare  walls.  The  ceiling  was  the 
source  of  a  diffuse  light. 

The  second  prototype  again  includes  just  the  VR  model  (with  kitchen  and  WC  connected 
by  a  corridor  and  with  several  typical  objects  -  bathtub,  light,  kitchen-sink  - 
automatically  generated  by  the  system).  This  prototype  had  two  different  types  of  light: 
natural  and  artificial.  Three  types  of  navigation  were  implemented:  free;  fixed  height;  and 
separate  view  points.. 

The  third  prototype  evaluations  included  all  three  modules.  As  the  first  two  modules  are 
typically  Windows  (point-and-click),  this  paper  only  reports  the  evaluations  made  of  the 
VR  module.  The  virtual  world  is  identical  to  the  last  prototype  (richer  in  objects)  and  the 
users  are  the  ones  that  design  the  rooms,  place  the  typical  objects  and  select  tile  patterns 
that  are  going  to  be  visualized  in  the  VR  model.  The  functionality  of  navigating 
backwards  was  added,  in  this  prototype. 

Development  process 

Design  team.  The  design  team  constitution  and  ergonomist  role  in  the  team  influenced 
the  design  and  development  processes  of  both  projects.  RECMUVI  design  team,  with 
three  computer  scientists,  three  ergonomists,  and  two  environmental  engineers  developed 
the  project  starting  from  computer  solutions. 

The  lack  of  comprehension  of  the  computer  scientist  ergonomists  role  on  the  design  team 
led  them  to  minimize  the  importance  of  heuristic  and  participative  evaluations.  In  the 
heuristic  evaluations  (always  a  posteriori  regarding  computer  implementation) 
suggestions  made  by  ergonomists  were  followed  only  when  they  were  easy  to  implement. 
Problems  with  software  architecture  were  systematical  ignored,  as  changes  in  structure 
pointed  to  a  radicaly  different  design  philosophy.  For  computer  scientists  interface  was 
secondary  regarding  to  programing  problems  and  solutions. 

In  the  participative  evaluations,  it  was  very  interesting  to  observe  the  computer  scientists 
change  of  behaviour  regarding  evaluation  sessions.  From  the  moment  they  saw  subjects 
fighting  with  their  design,  several  of  them  enthusiasticaly  adhered  to  the  evaluation 
sessions. 

TILE  VIZ  design  process  followed  a  completely  different  philosophy.  One  of  the  authors 
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was  the  manager  of  the  design  team,  and  that  radicaly  change  the  ergonomists  role.  Their 
participation  started  from  the  beginning  of  process  and  the  interface  was  the  first  thing  to 
be  defined.  Therefore  the  system  design  followed  the  interface  design.  The  goal  was  to 
simplify  and  potencialize  users’  performance. 

Almost  all  changes  proposed  by  ergonomists  to  other  team  members  (three  computer 
scientists)  were  accepted  and  implemented  in  the  system.  Participative  evaluations  were 
prepared  by  the  design  team  and  followed  with  attention  by  all  of  its  members. 

Methodology,  The  first  question  faced  by  the  evaluation  team  was  related  to  the  validity 
of  transfer  to  VR  systems  of  traditional  methodology  of  interface  evaluation.  Before  the 
option  for  traditional  methodology,  the  hypothesis  of  drama-based  evaluations  was 
considered  as  suggested  by  Brenda  Laurel  (1993),  but  those  dramatizations  were  not 
chosen  due  to  lack  of  resources. 

Therefore,  both  projects  were  subject  to  usability  iterative  tests  using  three  vertical 
prototypes  (Hix  &  Hartson,  1993).  In  both  projects  heuristic  evaluations  were  carried  out 
by  two  ergonomists  (2  evaluation  points  in  RECMUVI  and  a  continuous  evaluation  on 
TILE  VIZ)  and  two  tests  with  nine  non-expert  users.  Subject  numbers  per  session  are 
those  advised  in  literature  for  the  two  evaluation  types  (Nielsen,  1992;  Virzi,  1990, 
respectively). 

In  TILE  VIZ  there  was  a  third  evaluation  session  in  real  context  with  103  subjects  testing 
the  software  during  three  days. 

Design  evolutions 
RECMVVI 


Figure  3.  RECMUVI  -  Evaluations  Scheme. 


On  heuristic  evaluations  Ravden  and  Johnson  (1989)  usability  criteria  were  used  as 
guidance.  These  criteria  were  also  used  to  formulate  questions  during  the  evaluation 
sessions,  especially  in  the  3'“'^  prototype,  where  a  group  interview  took  place,  with  open 
questions. 

Subjects  had  15  minutes  to  explore  the  game.  Then  there  was  an  individual  interview  (2"^^ 
prototype)  or  group  interview  (3'^^  prototype). 

Some  objective  performance  indicators  were  chosen  and  developed  (e.g.  number  of 
picked  up  and  dropped  objects,  execution  times).  Users  were  asked  to  draw  the  virtual 
home  plan. 
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The  instruction  for  the  participative  evaluations  was  simple:  find  the  objects  and  drop 
them  into  the  correct  recipients.  All  the  interactions  were  taped  on  video  then  analyzed  by 
the  authors. 

On  the  participative  evaluation  made  to  the  2”^*  prototype  some  questions  were  asked 
during  the  interactions,  as  a  means  of  promoting  the  thinking-aloud  process.  But  both  the 
thinking- aloud  process  and  the  questions  revealed  to  be  concurrent  stimuli  that  interfere 
with  users  performance  (e.g.  interactions  stops). 

On  the  evaluation  of  the  prototype  the  thinking-aloud  technique  and  the  questions 
during  the  interactions  were  substituted  by  a  group  interview  a  less  intrusive  technique. 

TILE  VIZ 


Figure  4.  TELE  VIZ  —  Evaluations  Scheme. 


In  this  project,  authors  used  the  techniques  that  they  found  most  valid  in  RECMUVI.  The 
interactions  on  the  r‘  participative  evaluation  were  coded  in  a  grid  and  taped  on  video  for 
further  analysis;  after  the  interactions  users  were  interviewed  in  group. 

The  criteria  of  Ravden  and  Johnson  (1989)  were  again  used  for  extracting  useful 
information  fi:om  users.  Group  interviews  showed  to  be  a  good  means  of  explaining 
clearly  the  problems  that  the  subjects  found,  and  precious  ideas  generators. 

Some  improvements  were  introduced.  First,  the  heuristic  evaluations  were  performed 
continuously,  the  ergonomists  were  always  present  when  decisions  were  made  regarding 
the  interface  style  and  interactions  type. 

Second,  tasks  were  given  to  users.  On  the  T*  participative  evaluation,  those  tasks  were 
specific,  closely  related  to  vision  criteria  (accuracy,  color,  distance  measure)  and 
locomotion  (execution  styles  and  times).  On  the  2"^^  evaluation,  tasks  were  simply 
suggested,  since  the  context  required  such  an  approach,  and  object  manipulation  criteria 
were  added. 

It  should  be  noted  that  the  2"*^  evaluation  was  extremely  useful,  since  the  context  where  it 
took  place  was  similar  to  the  one  of  its  final  use.  Authors  were  able  to  see  the  impact  of 
context  on  system  and  users  performance. 

The  final  participative  evaluation  was  a  confirmation  for  the  choice  of  the  navigation  and 
manipulation  tool,  following  the  template  of  the  first  evaluation,  with  individual 
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interviews  and  without  video  recording. 


Design  Evolutions 

The  1  considerations  to  be  drawn  were  related  to  simple  indicators,  such  as  luminance, 
height/width  ratio,  that  can  be  considered  to  be  basic  problems  on  the  construction  of 
virtual  worlds.  Although  those  problems  seem  rather  evident,  one  should  not  take  into 
account  just  psychophysiologic  criteria,  but  also  consider  aesthetic  aspects. 

The  next  issue  is  the  necessity  to  implement  a  medium  height  of  navigation.  One  of  the 
central  elements  of  different  difficulty  stages  could  be  the  development  of  different  types 
of  navigation.  So,  in  TILE  VIZ,  the  second  prototype  includes  three  navigation  types,  that 
cover  all  system  needs.  In  general,  it  seems  that  fixed  height  navigation  and  the  free 
navigation  require  different  cognitive  resources,  and  this  can  be  applied  to  all  VR 
systems. 

The  search  for  a  better  navigation  system  can  be  viewed  with  the  change  introduced  in  2"^^ 
to  3^^*  prototype  of  RECMUVI.  The  dissociation  between  viewpoint  and  moving  direction 
implied  the  existence  of  two  different  sets  of  visual  cues.  On  one  hand  we  have  the 
images  actualization  in  response  to  head  movements.  On  the  other  hand,  we  have  the 
modifications  on  direction  arrow  in  response  to  hand  movements. 

This  solution  introduced  a  greater  complexity  level,  since  users  would  have  to  control 
two  position  referentials,  each  one  with  their  visual  cues.  Furthermore,  the  direction 
arrow  is  “linked”  to  the  viewpoint  since  subject  is  always  seeing  the  arrow,  whatever  the 
direction  he  is  looking  to;  finally,  the  arrow  takes  the  relative  position  of  the  direction 
(hand  movement)  and  coordinates  with  the  eyeing  direction.  So  they  are  not  completely 
independent. 

The  complexity  of  such  solution  should  have  been  studied,  introducing  his  characteristics 
gradually,  allowing  an  evaluation  of  the  cognitive  capacities  required. 

The  next  point  reports  the  inclusion  of  visual  cues  to  guide  the  user  in  the  virtual  world. 
One  first  question  concerns  the  reduction  of  the  visual  field  and  the  discontinuity  on  the 
space  apprehension,  requiring  bigger  memory  resources  to  its  representation.  A  simple 
corridor  with  6  different  doors  demands  that  the  subject  explore  systematically  the 
different  parts  that  constitute  that  corridor,  not  apprehending  it  with  one  glance.  The 
objects  that  go  out  of  the  limited  visual  field  have  to  be  retained  in  memory,  especially 
their  locations.  These  affirmations  are,  in  part,  supported  by  data  obtained  through  plant 
drawings  in  RECMUVI  project.  After  the  corridor  doors  are  opened,  subjects  present 
more  correct  plans,  although  the  navigation  is  much  more  difficult  and  the  exploring  is 
then  restricted  (note  that  the  subjects  in  the  3*^^  prototype  enter  less  in  the  different 
rooms). 
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Table  L  Comparative  results  of  RECMUVI  i 

orototypes  2  e  3. 

Mean 

Frequency  of 
Picked 
Objects 

Mean 

Frequency  of 
Dropped 
Objects 

Mean  Number 
of  Correct 
Spatial 
Identification 

Mean 
Number  of 
Correct 
Rooms 

Average  of 
Entries 

Prototype 

2 

3 

2,5 

4 

3 

5,2 

Prototype 

3 

1.8 

0,8 

4,5 

4,3 

4,5 

The  other  issue  in  visual  cues  has  to  do  with  the  difficulty  to  calculate  distances  in  the 
virtual  environment.  The  nature  of  the  TILE  VIZ  itself  solved  the  problem,  since  the 
simulation  of  tile  application  introduces  a  squad  in  the  space  that  facilitates  the 
navigation. 

Finally,  the  last  thing  to  report  is  the  change  of  the  navigation  tool,  from  the  trigger  to  a 
pad  in  TILE  VIZ.  On  one  hand,  this  decision  was  taken  due  to  the  end  increase  system 
functions.  On  the  other,  it  was  essential  to  implement  the  functionality  of  walking 
backwards. 

Walking  backwards  is  cmcial  since  users  had  difficulties  in  executing  a  rotation  of  180 
degrees,  necessary  for  driving  away  from  an  object,  for  instance.  The  pad  allowed  the 
existence  of  one  button  to  move  forward  and  another  to  move  backwards. 

Finally,  since  this  pad  is  an  infrared  one,  users  showed  less  navigation  difficulties,  as 
wires,  that  were  a  constant  embarassment  were  reduced. 

Conclusions 

The  approaches  concerned  with  the  usability  iterative  tests  are  accepted  to  be  the  most 
effective,  as  suggested  by  Landauer  (1988),  Jeffries,  Miller,  Wharton,  and  Uyeda  (1991), 
Bayley,  Allan  and  Raiello  (1992)  and  Virzi,  Sorce  e  Herbert  (1993).  The  application  of 
heuristic  evaluations  and  end  users  tests,  as  complementary  methodologies,  enables  the 
correction  of  the  gross  majority  of  system  problems.  TILE  VIZ  project  experience  was 
truly  helpful  in  clarifying  this  issue. 

Barnard  et  al.  (1986,  in  Barnard,  1991)  raise  doubts  about  the  use  of  thinking-aloud 
techniques.  On  a  first  contact  with  a  virtual  environment,  the  interaction  developed  by  the 
user  is  centered  on  action  itself.  Users  search  for  correlations  between  body  movements 
and  the  system  states,  paying  less  attention  to  verbal  behaviors. 

As  the  thinking-aloud  technique  demands  cognitive  resources  needed  for  the  interaction, 
it  affects  users  ability  to  explore  the  virtual  world  (RECMUVI).  This  technique  may  be 
admitted  as  a  good  double  task  indicator. 

Task  definitions  and  group  interviews  revealed  to  be  more  productive  techniques.  On  one 
hand,  navigation  and  objects  manipulation  problems  are  clarified  with  task  performances, 
evaluated  with  objective  criteria.  On  the  other,  group  interviews  enable  a  sharing  of 
individual  experiences,  a  discussion  about  the  central  aspects  of  the  interaction  and 
proposals  of  solutions  for  recognized  problems. 
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Another  important  problem  raised  by  Bailey  et  al.  (1992),  is  concerned  with  the  abilities 
that  should  characterize  the  evaluator.  This  does  not  seems  to  restrict  to  different  levels  of 
technical  knowledge  about  user  interfaces,  or  systems,  in  spite  of  the  importance  it  surely 
has  (Nielsen,  1992).  Barnard  (1991)  when  referring  that  a  significant  share  of  die 
exploratory  development  methodologies  depends  on  the  evaluators  intuition  and  expertise 
qualifications  also  sustain  this  fact. 

Future  research  studies  will  focus  on  a  VR  interfaces  development  model  with  a 
sequential  implementation  of  the  characteristics  and  a  recording  of  users*  behaviors  in 
minimal  units. 

Let  us  now  look  at  VR  issues.  Following  some  ideas  proposed  by  Varela  (1995)  it  can  be 
accepted  that  a  cognitive  system  experience  implies  a  linkage  between  a  structure  and  an 
environment.  Therefore,  there  is  a  context  where  the  user,  as  a  cognitive  entity  with  a 
body,  possess  one  perspective  and  relates  to  the  VR  system  through  a  specific  dynamic. 

As  is  viewed  by  Steuer  (1992,  in  Slater  &  Usoh,  1993)  VR  is  created  by  individual 
consciousness  mechanisms,  revealing  the  importance  of  studying  subjective  variables  and 
their  effects  on  the  interaction  of  the  subject  with  the  virtual  world  (e.g.,  presence 
feehng). 

It  is  easy  to  create  a  virtual  world  sensation  of  presence  because  human  cogmtive  system 
generates  the  illusion  of  immersion.  This  implies  that,  in  a  VR  system,  a  careful 
conception  of  all  aspects  related  with  the  interaction  (which  comprises  vision,  navigation 
and  object  manipulation)  should  be  developed,  especially  when  deahng  with  non-expert 
users.  It  follows  that,  if  the  feeling  of  presence  is  easy  to  obtain  in  a  first  stage,  helped  by 
the  technology  natural  attractiveness,  it  is  clear  that  the  following  interactions,  if  not 
intuitive  and  easy  to  learn,  may  kill  the  first  impression. 

Keeping  the  exciting  aspect  of  VR  demands  intuitive  and  easy  to  learn  systems  able  to 
generate  comfort  and  safety  feelings  corresponding  to  users  expectations.  To  accomphsh 
that,  users  should  get  an  added  value  from  the  interactions,  and  well  defined  objectives 
should  be  attained  regarding  learning,  entertainment  and  decision-making. 
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Abstract 

This  paper  will  present  a  project,  which  was  first  initated  in  1994  as  a  graduate 
students  seminar  and  is  now  being  continued  as  a  research  project  in  a  cooperation  of 
computer  scientists,  architects  and  archaeologists.  An  ancient  roman  city  (Colonia 
Ulpia  Traiana  near  todays  Xanten  in  Germany)  has  been  reconstructed,  using  various 
levels  of  abstraction.  On  the  coarsest  level,  a  3D-model  of  the  whole  city  was 
established,  distinguishing  between  different  historical  periods  of  the  city.  The  second 
level  picks  places  of  special  interest  (temples,  the  forum,  the  amphitheater,  the  town- 
baths  etc.)  and  reconstructs  these  buildings  or  groups  of  buildings.  On  the  finest  level 
important  interior  parts  or  functional  details  like  the  Hypocaustae  in  the  town-baths 
are  modelled.  All  econstructions  are  oriented  as  close  as  possible  to  results  from 
excavations  or  other  available  documents. 

All  levels  of  the  3D-model  have  been  visualized  using  photorealistic  images  and 
sequences  of  video  animations.  The  3D  model  is  integrated  into  a  multimedia 
environment,  augmenting  the  visualization  elements  with  plans  of  the  city  and 
individual  buildings  and  with  text  documents.  It  is  intended,  that  parts  of  the  outlined 
system  will  be  available  at  the  site  of  the  ancient  city,  where  today  a  large  public 
archaeological  park  is  located. 

Introduction 

Museums  are  in  many  aspects  a  very  suitable  field  of  application  for  multimedia 
techniques.  This  contibution  will  focus  on  public  information  systems,  which  can  be 
used  as  multimedia  guides  on  CD-ROM  or  in  a  communication  network  as  well  as  in 
the  museum  itself.  There  are  already  a  couple  of  examples  available  in  internet,  e.g.  a 
guide  through  the  National  Air  and  Space  Museum  in  Washington  D.C.  (NASM 
1995)  with  many  image  and  text  documents  or  through  the  Museum  of  History  of 
Science,  Florence  (Bemi  and  Guidi  1995),  Applications  in  the  museum  itself  range 
from  orientation  guides  (Where  can  I  find  which  department,  which  exhibit  70  to 
multimedia  informations  at  individual  objects.  Significant  advantages  over  standard 
information  charts  and  audio  tapes  arise  especially,  if  visualization  of  dynamic 
processes  (e.g.  function  or  manufacturing  of  technical  exhibits)  support  the  visitors 
understanding.  Moreover,  an  interested  user  can  be  supplied  with  nearly  arbitrarily 
much  information  on  details  or  on  background. 

Using  the  opportunity  to  individually  select  information,  a  visitor  can  thus  play  a 
much  more  active  role  in  a  multimedia  supported  exhibition  than  in  classical 
concepts,  where  he  is  usually  restricted  to  pure  passive  examination.  The  most 
attractive  field  of  application  of  multimedia  systems  in  museums  may  yet  be  all  those 
areas,  where  objects  being  otherwise  invisible  can  be  shown. 
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An  example  for  this  application,  a  prototype  of  a  multimedia  archaeological  museum, 
is  presented  in  this  paper. 

In  a  graduate  students  seminar  with  the  title  ’Seeing  the  invisible  reconstruction  of  an 
antique  city  by  CAD"  16  students  of  the  Department  of  Architecture  and  Civil 
Engineering  at  the  University 

of  Dortmund  created,  in  cooperation  with  computer  scientists  and  archaeologists,  a 
huge  3D  model  of  the  ancient  roman  city  'Colonia  Ulpia  Traiana’  near  todays  Xanten 
in  Germany.  A  prototype  for  this  seminar  was  Koobs  pioneering  reconstruction  of  the 
Cathedral  of  Cluny 

(Cramer  and  Koob  1993).  The  primary  goal  of  the  seminar  was  to  teach  students  in 
3D-CAD  modelling  and  visualization.  They  were  challenged  by  the  required 
modelling  of  very  complex  geometric  objects  and  were,  on  the  other  hand,  given 
enough  freedom  to  use  and  to  develop  individual  knowledge  (and  intuition)  about 
classical  culture  and  buildings.  In  examining  processes  of  construction,  students  had 
to  study  thoroughly  principles  and  techniques  of  anchient  master  builders.  Thus,  the 
use  of  CAD  contributed  much  more  to  the  architectural  education  of  the  students  than 
only  to  the  development  of  technical  skills. 

Our  CAD  model  uses  different  levels  of  abstraction.  On  the  coarsest  level,  a  model  of 
the  whole  city  was  established,  giving  the  frame  for  the  second  level,  where  places  of 
special  interest  (several  temples,  the  amphitheater,  the  town-baths  etc.)  were 
elaborated.  On  the  third  level,  important  interior  parts  or  functional  details  were 
modelled.  All  reconstructions  were  orientated  as  close  as  possible  to  results  of 
excavations  or  to  other  available  historical  documents. 

The  integration  of  the  CAD-model  in  a  multimedia  document  and  an  example 
session  of  a  'tour'  through  the  document  will  be  outlined  in  the  next  sections. 

System  design  and  implementation 

The  implemented  'multimedia  museum’  (Rank  1 995)  is  based  on  the  3D  CAD-model, 
having  been  created  using  a  commercially  available  CAAD-system  (Nemetschek 
1994)  with  powerful  solid  modelling  capabilities. 

Within  this  system,  a  large  number  of  photorealistic  images,  using  standard  shading 
or  ray-tracing  techniques,  was  created  and  stored  as  gif-  or  jpeg-files.  It  was  also 
possible  to  create  video- sequences  of  selected  parts  of  the  model  and  to  store  these 
clips  as  mpeg-files.  Images  and  videos 

are  integrated  in  a  multimedia  system,  using  html-version2- standard  and  public 
domain  tools  for  browsing  and  visualisation.  The  overall  structure  of  the  document  is, 
according  to  the  structure  of  the  underlying  CAD-model  hierachical,  with  some 
additional  horizontal  links  between  particular  parts  of  the  document. 

We  consider  it  especially  important  to  design  the  structure  of  a  multimedia  document 
ver>'  carefully,  so  that  a  user  of  the  document  is  able  to  understand  this  structure 
quickly  and  intuitively.  For  our  application  it  is  very  natural  to  use  images  of  the 
model  as  basic  ordering  elements,  applying  the  concept  of  'clickable  maps'  to  select 


information  on  the  next  level  of  hierarchy.  Considering  for  example  the  root  -level  of 
the  document,  a  birds  eye  view  of  the  city  is  presented,  and  the  user  can  select 
information  on  different  buildings,  clicking  on  these  objects  in  the  images. 

Moreover,  a  unique  box  with  available  documents  (floor  plans,  index,  videos)  is 
available  on  every  layer  of  the  document,  preventing  the  user  from  ’getting  lost  in  the 
large  amount  of  informations. 

Examples 

Since  its  opening  in  1977  more  than  4.5  million  people  have  visited  the 
Archaeological  Park  Xanten.  The  ancient  monument  Colonia  Ulpia  Traiana  is  open  to 
scientific  investigation,  but  in  particular  is  presented  to  the  public.  Methods  and 
results  of  archaeological  research  are  displayed  at  the  spot  where  history  took  place 
(from  Rieche  1994).  The  design  of  the  multimedia  guide,  being  aligned  to  the  general 
goals  of  the  APX,  is  intended  to  serve  as  well  as  a  first  introduction  to  the  Park  and 
also  as  a  system  to  provide  background  information. 

Both  ways  of  use  start  entering  the  system  in  a  page  shown  in  Figure  1,  being 
composed  of  first  textual  information  and  a  perspective  view  of  the  city. 
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Figure  1  Clickable  image  of  the  whole  Roman  city 
Selecting  e.g.  the  amphitheater  or  the  town-baths  in  the  clickable  map  leads  to 
branches  with  images  and  additional  text  information  partially  shown  in  Figures  2 
and  3, 
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Figure  2  Amphitheater  with  image  of  construction  details 
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Figure  3  The  town-baths  with  technical  informations  about  the  Hypocaustae 


Discussing  the  branch  assoziated  to  the  capitoline  temple  it  will  be  shown  that  a 
multimedia  system  based  on  a  3D  CAD-model  can  easily  provide  a  lot  of  additional 
information,  being  only  implicitly  related  to  the  spacial  model.  The  entrance  point 
into  this  object  is  again  a  photorealistic  image  of  the  temple  providing  options  to 
select  more  images  in  details  (Figure  4).  Also  available  is  a  floor  plan  (Figure  5) 
together  with  a  scanned  picture  of  the  (very  small)  part  of  the  fully  excavated  site 
(Figure  6),  This  part  is  marked  in  the  floor  plan  (shaded  area)  and  thus  gives  the  user 
a  very  immediate  idea  of  the  size  of  the  whole  site  of  this  temple.  Other  details  are 
shown  in  Figure  7,  where,  derived  from  the  3D-model,  construction  parts  of  the  front 
side  and  of  the  columns  are  plotted  and  explained  in  more  detail.  This  image  is  used 
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again  as  a  clickable  map  to  obtain  perspective  views  of  constitutional  elements  like  a 
part  of  the  roof  construction  with  capitals,  shown  in  Figure  8  as  CAD-model  and  as  a 
1:1  reconstruction. 
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Figure  5  The  capitoline  temple:  Floor  plan  with  shaded  excavated  area 


Conclusions  and  future  work 

Based  on  a  huge  3D  CAD-model  a  prototype  for  a  multimedia  system  to  be  used  as  a 
guide  through  an  archaeological  site  has  been  outlined.  The  system  offers  opportunities 
to  retrieve  information,  which  go  far  beyond  the  possibilities  of  classical  media.  Up  to 
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now,  the  system  is  composed  of  text,  high  resolution  photorealistic  images,  plans, 
photographs  and  sketches. 


Figure  8  A  detail  of  the  CAD-model  (left)  and  Photo  of  a  1:1  reconstruction  (right) 


With  the  availability  of  sufficiently  powerful  hardware  it  will  be  possible  to  allow  a  user 
an  online  animation  of  the  full  3D-model,  giving  him  the  opportunity  of  a  Virtual  walk' 
through  the  ancient  city.  Additionally,  parts  of  the  model  have  already  been 
implemented  on  a  '3D  workbench'  (Krueger  and  Froelich,  1994),  presenting  the  model 
with  the  possibility  of  online  interaction  on  a  'table'  in  stereo  view.  The  most  attractive 
perspective  of  the  multimedia  system  is  yet  a  presentation  at  the  site  of  the  ancient  city 
itself,  where  today  a  large  public  archaeological  park  is  located.  The  viscinity  of 'real' 
excavation  and  'virtual'  reconstruction  will  give  the  visitor  a  much  better  chance  to 
understand  the  site  and  the  roman  culture  as  a  whole  than  any  classical  way  of 
presentation  of  archaeology. 

A  cknowledgement: 

We  would  like  to  thank  the  APX,  especially  Dr.Precht  and  Dr.Zieling  as  well  as  the 
Nemetschek  Programm  GmbH  for  their  support  to  this  project. 

References 

NASM  1995.  National  Air  and  Space  Museum,  http://www.nasm.edu/ 


Berni,  M.,  Guidi,  F.  1995.  Home  page  of  the  Institute  and  Museum  of  History  of 
Science,  Florence  ITALY,  http://galileo.imss.firenze.it/ 

Rank,  E.  1995.  Ein  multimediales  archaeologisches  Museum . 
http :  //www.  bau  wesen/forschung/xanten/xanten .  html . 


363 


Cramer,  H.,  Koob,  M.,  1993,  Cluny  -  Architektur  als  Vision,  Edition  Braus, 
Heidelberg 


Nemetschek  Programmsystem  GmbH,  1994,  Allplan  Version  JO.  Muenchen, 

Rieche,  A,  1994.  Guide  Through  the  Archaeolocical  ParkXanten.  Rheinland 
Verlag  GmbH,  Kdln. 

Krueger,  W„  Froelich,  B.,  1994.  The  Responsive  Workbench.  IEEE  Computer 
Graphics  and  Applications. 


364 


Subjective  Realism  in  a  Simulated  Squash  Game 


Heiko  Hecht 

Zentrumfur  interdisziplindre  Forschung 
Universitdt  Bielefeld 
D-33619  Bielefeld 


Abstract 

Subjective  realism  is  thought  to  play  an  important  role  in  computer  simulation.  The 
purpose  of  several  experiments  was  to  assess  to  what  degree  subjective  realism  can  be 
captured  by  well  defined,  task-relevant  information  such  as  time  delays  or  resolution,  and 
to  what  degree  it  has  to  be  considered  a  psychological  concept  in  its  own  right.  We 
investigated  immersive  variables  that  were  manipulated  above  and  beyond  the  visual 
information  necessary  to  carry  out  the  task.  Three  experiments  were  conducted  to 
compare  performance  measures  with  ratings  of  subjective  realism  within  an  engaging 
and  demanding  task.  Observers  were  confronted  with  a  simulated  squash  court. 
Accuracy  of  racket  placement  and  timing  were  assessed  as  a  function  of  gratuitous  cues 
for  compellingness  (such  as  depth  cueing,  cast  shadows)  and  as  a  function  of  objective 
disturbances  in  the  ball’s  trajectory.  Subjective  realism  proved  to  be  a  psychological 
variable  in  its  own  right  that  cannot  be  predicted  by  task  performance.  Pragmatic,  task- 
specific  criteria  were  found  to  be  particularly  important  determinants  of  subjective 
realism. 

Introduction 

Subjective  realism  has  been  acknowledged  to  be  a  major  aspect  in  the  evaluation  of 
visual  simulations.  The  goal  of  this  paper  is  to  explore  status  of  subjective  realism  form 
a  psychological  point  of  view.  1  demonstrated  that  subjective  realism  cannot  be  reduced 
to  performance  measures,  that  stimulus  properties  influencing  compellingness  should  not 
be  studied  in  isolation,  and  that  subjective  realism  bears  a  complex,  non-intuitive 
relationship  with  stimulus  parameters  of  the  simulation. 

A  taxonomy  for  testing  reality  or  realism  has  been  proposed  by  Kruse  and  Stadler  (1990), 
who  have  attempted  to  establish  a  catalog  of  criteria  that  contribute  to  phenomenal 
realism.  They  suggest  three  broad  categories  of  criteria  for  reality:  (1)  Syntactic  criteria, 
such  as  contrast,  starkness  of  object  contours,  richness  of  texture,  three-dimensionality, 
intermodal  congruence,  motion,  and  spatial  location.  All  primary  and  secondaiy  depth 
cues  would  fall  into  this  category.  (2)  Semantic  criteria:  Significance,  expressiveness, 
and  context  congruity  are  addressed  here.  For  example,  a  perfectly  weU-rendered  object 
may  not  look  compelling  because  it  is  placed  in  an  inappropriate  context.  (3)  Pragmatic 
criteria  are  the  possibility  of  the  object  to  become  part  of  an  action,  the  degree  to  which 
the  event  can  be  anticipated,  and  inter- subjective  agreement  on  the  object.  An  artificial 
situation,  even  if  it  does  not  attempt  to  replicate  a  maximal  number  of  aspects  associated 
with  a  real-world  experience  may  be  very  compelling  if  it  becomes  a  meaningful  part  of 
an  action.  To  be  able  to  differentiate  between  and  separately  manipulate  syntactic, 
semantic,  and  pragmatic  aspects  of  subjective  realism,  a  squash  game  was  simulated  on  a 
computer. 
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Experimental  Setup 

A  simulation  of  a  squash  game  was  chosen  and  reduced  to  the  following  action  sequence 
during  the  experimental  trials:  A  ball  was  simulated  to  fly  toward  the  far  wall  of  the  court 
(as  if  hit  by  an  invisible  opponent),  rebound,  and  then  move  toward  the  observer.  By 
manipulating  the  mouse,  the  observer  could  move  a  simulated  racket  and  "hit"  the 
oncoming  ball.  Independent  variables  were  shadow,  amount  of  detail,  texture,  fog,  and 
sound  effects.  Pragmatic  variables  were  success  rate  as  manipulated  by  racket  size, 
random  path  deflections  and  delayed  mouse  action.  On  all  trials,  positioning  of  the 
racket  and  timing  of  the  action  were  used  as  performance  measures,  a  compellingness 
rating  was  added  to  directly  assess  subjective  realism. 

The  stimuli  were  generated  on  a  Silicon  Graphics  Indigo  2  XZ  workstation  with  a 
resolution  of  1280  xl024  pixels  and  a  refresh  rate  of  72  Hz  (non-interlaced).  The 
animation  update  rate  was  18  frames/second.  Observers  sat  with  their  eye-point  27  cm 
away  from  the  screen  such  that  their  line-of- sight  was  centered  on  the  20"  display  screen 
The  monitor  was  38  cm  wide  and  29  cm  high  resulting  in  a  visual  field  of  70°  by  56°. 
The  stimulus  displays  consisted  of  a  3-D  rendition  of  a  squash  court.  A  ball  would  fly 
toward  the  back  wall  as  if  hit  by  an  invisible  opponent.  After  it  rebounded  it  moved 
toward  the  observer,  whose  task  was  to  intercept  it  with  her/his  racket.  The  racket  could 
only  be  moved  on  the  invisible  front  wall  but  not  in  depth  (2-D  movement)  by  using  the 
computer  mouse.  The  viewpoint  of  the  observer  was  fixed  thus  simulating  a  stationary 
observer  with  a  very  long  arm.  A  thin  line  was  drawn  from  the  edge  of  the  racket  to  the 
floor  to  provide  additional  visual  support  for  the  fact  that  the  racket  never  moved  into  the 
court  but  remained  on  the  front  wall. 

Various  Experimental  Manipulations 

First  we  probed  into  some  parameters  that  are  critical  for  a  hitting  task  as  well  as  into 
some  parameters  that  offer  more  gratuitous  embellishment,  such  as  sound  and  lighting. 
To  provide  a  standard  to  anchor  judgments  of  subjective  realism,  some  clearly  unrealistic 
changes  were  introduced  in  some  trials:  The  ball  could  deviate  from  its  motion  path  in 
mid  air,  as  if  deflected  by  an  unseen  force.  A  large  number  of  syntactic  and  semantic 
criteria  were  varied,  while  the  task  remained  always  identical.  Surprisingly,  some 
manipulations  that  were  thought  to  have  a  strong  influence  on  performance  (e.  g.  the  cast 
shadow  of  the  ball)  only  affected  judged  realism,  and  vice  versa. 

Also,  the  consequences  of  the  action  were  manipulated.  Not  the  whole  racket,  but  only  a 
cross  hair  indicating  its  center  was  visible.  At  the  same  time  the  actual  size  of  the 
invisible  racket  was  varied.  Thus,  the  number  of  successful  hits  could  be  changed 
independently  from  parameters  of  the  simulation.  The  success  of  the  action,  above  and 
beyond  all  other  manipulations,  had  a  strong  positive  effect  on  judged  realism. 

Both  sound  and  the  shadow  cast  by  the  ball  produced  an  informative  asymmetry  between 
performance  and  realism.  Sound  effects  were  not  exploited  to  fine-tune  action,  not  even 
to  reduce  timing  errors.  However,  trials  with  sound  effects  were  judged  to  be  more 
reahstic  than  those  without.  Shadow  had  the  opposite  effect:  It  increased  the  racket 
positioning  error  but  had  no  effect  on  judged  realism.  The  fact  that  shadow  distracted  in 
the  proper  positioning  of  the  racket  without  affecting  hit  rates  or  timing  of  the  ball 
suggests  that  shadow  was  used  to  fine  tune  the  racket  position,  albeit  with  negative 
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effects.  Shadows  are  known  to  exert  a  strong  influence  on  the  perceived  depths  of 
objects  and  may  even  override  other  depth  cues  and  create  powerful  illusions  (Kersten, 
Tarr,  &  BUlthoff,  1995).  Thus,  the  negative  shadow  effect  is  in  accordance  with  the 
literature. 

Observers  did  not  receive  explicit  feedback,  but  they  could  use  the  effect  of  their  action 
(hit  versus  miss)  to  judge  the  positioning  accuracy  of  their  action.  Tuning  errors,  on  the 
other  hand,  could  not  be  perceived  at  all.  We  considered  making  a  successful  hit  and 
rebound  dependent  and  correct  positioning  and  timing,  but  pilot  subjects,  including  the 
experimenter,  found  it  very  frustrating.  Also,  it  should  be  avoided  that  the  ball  flew 
through  the  racket  in  order  not  to  create  another  variable  affecting  realism.  The  fact  that 
variability  in  performance  was  carried  by  positioning  was  a  function  of  focusing  the 
observer’s  attention  on  hitting  the  ball,  which  was  much  harder  than  in  Experiment  1. 
This  effect  was  desirable  since  timing  errors  might  have  been  hard  to  interpret  given  the 
rather  slow  update  rate  of  13  Hz. 

Changing  the  light  transmittance  inside  the  squash  court  (fog)  produced  no  significant 
effects,  neither  did  it  affect  performance  nor  judged  realism.  This  indicates  that 
observers  could  do  without  the  information  contained  in  the  initial  phase  of  the  balls 
trajectory  after  leaving  the  rear  wall.  They  might  have  extrapolated  the  ball's  motion 
during  the  period  where  it  practically  merged  with  the  dark  background.  It  seems 
plausible,  that  observers  considered  fog  as  a  natural  effect  while  they  treated  pervasive 
effects  like  sound  as  necessary  for  a  realistic  rendition. 

In  sum,  the  decoupling  of  performance  and  judged  realism  in  the  case  of  sound  and 
shadow  shows  that  the  former  interact  in  non-trivial  ways.  It  is  thus  unhkely,  that  any 
given  performance  measure  is  able  to  capture  reahsm. 

In  another  experiment,  the  pragmatic  factor  was  more  thoroughly  explored  by  making  it 
independent  of  parameters  of  the  simulation.  This  was  achieved  by  making  the  racket 
invisible  except  for  a  cross-hair  indicating  its  center.  The  size  of  the  invisible  racket  was 
changed  unpredictably  from  trial  to  trial.  That  is,  a  well  executed  placement  of  the  racket 
could  lead  to  failure  to  hit  the  ball  (as  well  as  to  success);  and  a  sloppily  executed  racket 
placement  could  lead  to  a  success  (as  well  as  to  a  miss).  Success  rate  was  thus  decoupled 
from  quality  of  the  animation,  precision  of  the  action,  and  other  possible  confounds.  In 
all  other  respects  the  experimental  setup  remained  unchanged  with  the  exception  that  the 
shadow  manipulation  was  dropped,  mouse  action  was  always  instantaneous,  and 
visibility  was  changed  by  introducing  fog  to  the  display  on  some  trials. 

Manipulating  the  success  rate  of  the  motor  response  had  the  hypothesized  effects  (see 
Figure  1),  The  pragmatic  factor  of  success  exerted  a  strong  influence  on  judged  reahsm. 
Trials  in  which  balls  rebounded  from  the  invisible  racket  looked  more  natural  and 
realistic  to  observers  compared  to  trials  where  the  ball  went  past  the  racket.  This  effect 
cannot  be  attributed  to  any  syntactic  or  context  effects  of  the  simulation  but  solely  on  the 
outcome  of  the  action.  A  roughly  linear  trend  between  judged  realism  and  racket  size 
indicates  that,  ceteris  paribus,  reahsm  is  a  more  or  less  linear  function  of  success. 
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Figure  1.  Mean  judged  realism  as  a  function  of  racket  size.  Note  that  the  racket  was 
invisible  and  displays  were  identical  in  all  cases.  The  rackets  center  was  marked  by  a 
cross  hair  and  could  be  10  %,  200  %,  or  600  %  of  its  standard  size.  Timing  and 
positioning  accuracy  were  identical  in  these  conditions.  Thus,  identical  actions  based 
on  identical  visual  stimulation  yield  different  realism  as  a  function  of  their  outcome. 
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Discussion 

The  results  support  the  notion  that  judged  realism  can  neither  be  confined  to  pure 
performance  measures  nor  to  display  parameters  of  the  simulation.  In  addition  to  these 
factors,  the  observer’s  action  context  and  the  outcome  of  the  event  have  been  shown  to  be 
critical  for  the  degree  of  subjective  realism  in  the  simulated  scene.  In  hindsight,  these 
results  justify  the  unified  measure  of  subjective  realism  that  was  employed. 

When  confronted  with  the  task  to  compare  two  pictures  or  two  computer  simulations,  it 
is  hard  to  imagine  a  difference  between  them  that  does  not  also  reflect  a  difference  in 
realism.  For  instance,  if  an  observer  is  confronted  with  two  displays  that  vary  in  only 
one  aspect,  let’s  say  stereo,  and  if  he/she  is  then  asked  which  display  was  nicer  or  more 
realistic,  the  display  with  stereo  is  usually  preferred  (e.  g.  Hendrix  &  Barfield,  1996). 
However,  by  itself,  this  judgment  is  rather  worthless  because  a  different  reference  object 
or  reference  class  can  completely  change  the  judgment.  Moreover,  if  stimuli  vary  on  one 
dimension  only  then  the  observer  is  maximally  constrained  and  will  therefore  accept 
almost  any  label  for  the  difference.  One  could  probably  replace  realistic  with  words  like 
cozy,  strong,  or  heavy  and  still  get  the  same  results.  Thus,  to  interpret  the  results  as  a 
difference  in  judged  realism  an  additional  argument  has  to  be  made  to  connect  the  rating 
to  a  content. 

As  Gestalt  psychologists  have  shown,  this  depends  on  the  hierarchical  nesting  of  the 
stimuli  in  their  context.  From  our  results  it  is  clear,  that  factors  such  as  the  success  of  the 
action  are  part  of  this  context.  It  is  therefore  questionable  if  much  is  learned  if  only  one 
or  two  parameters  are  varied  simultaneously.  The  concept  of  realism  appears  to  be  too 
complex  to  be  fully  grasped  by  studying  aspects  of  it  in  isolation.  Thus,  in  the  case  of 
subjective  realism,  complex  experimental  situations  may  well  be  a  virtue  rather  than  a 
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vice.  At  the  same  time,  results  obtained  with  small  parameter  sets  have  to  be  interpreted 
very  carefully.  The  rating  scales  that  were  used  in  the  present  experiments  did  not  rely 
on  a  small  set  of  parameters,  but  they  reflect  considerable  variability  in  syntactic, 
semantic,  and  pragmatic  aspects  of  the  action  and  its  simulated  environment.  The  fact 
that  they  produced  interpretable  results  justifies  the  choice  of  a  rating  scale  that  required 
a  single  rating  after  each  trial.  Moreover,  the  simultaneous  significant  effects  of  several 
independent  variables  (sound,  success,  mouse  delay,  and  path  deflection)  on  judged 
realism  indicates  that  observers  do  not  just  focus  on  one  salient  aspect  of  the  simulation, 
but  that  they  do  integrate  -  knowingly  or  not  -  many  aspects  of  the  display  into  one 
meaningful  judgment.  The  lack  of  significant  interactions  between  the  different 
independent  variables  revealed  that  the  resolution  of  the  compound  judgment  was  limited 
to  main  effects. 

Possible  alternatives  to  the  realism  ratings  could  have  been  (1)  cue  trade-off  studies  and 
(2)  successive  addition  models.  Trade-off  studies  place  two  cues  in  conflict.  The  cue 
that  wins  is  the  stronger  one.  Underlying  this  model  is  a  static  view  of  cue  strength  that 
is  very  situation  specific.  As  a  function  of  this  model  typical  empirical  results  are  that, 
depending  on  situation  and  observer,  different  cues  win.  However,  these  methods  could 
not  be  used  here  for  two  reasons.  First,  too  httle  is  known  about  variables  or  cues  that 
contribute  to  subjective  realism.  Second,  compellingness  or  realism  are  integral  concepts 
that  have  to  be  assessed  in  a  complex  (non-additive)  fashion.  This  was  achieved  in  the 
present  study,  since  no  explicit  understanding  of  the  concept  of  realism  was  necessary  for 
the  task.  The  required  rating  scale  of  compellingness  or  realism  was  anchored  in  the  first 
trial  blocks  relative  to  the  variability  in  the  stimuli 

In  sum,  the  importance  of  pragmatic  criteria  for  subjective  realism  has  been 
demonstrated.  Time  delay  of  consequences  caused  by  the  motor  action  (mouse 
manipulation)  had  a  strong  negative  effect  on  judged  realism.  Likewise  did  outcome  of 
the  action  (hit  or  miss)  influence  judged  realism.  This  entails  that  it  is  not  very  useful  to 
define  subjective  realism,  or  presence  for  that  matter,  as  a  dimension  that  is  independent 
of  interactivity  or  that  does  not  play  a  role  in  standard  displays,  as  was  suggested  by 
Zeltzer  (1992). 

Conclusions 

The  findings  suggest  that  the  taxonomy  devised  by  Kruse  &  Stadler  (1990)  is  very  useful 
when  evaluating  subjective  realism  of  visual  simulations.  Moreover,  the  results  are  in 
agreement  with  hypotheses  by  Loomis  (1992)  that  distal  attribution  of  events  is  crucial  in 
subjective  realism.  The  results  also  are  compatible  with  and  extend  findings  by  Welch  et 
al.  (1996).  The  particular  implications  for  our  thinking  about  subjective  realism  can  be 
summarized  as  follows: 


1 .  Subjective  realism  cannot  be  reduced  to  the  fidelity  of  the 
graphical  rendition 

2.  Subjective  realism  cannot  be  reduced  to  performance  measures 

3.  Violations  of  physical  laws  often  do  not  affect  subjective  realism 
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4.  Subjective  realism  is  context  specific 

5.  Subjective  realism  is  highly  dependent  on  the  action 

the  observer  is  involved  in  as  well  as  the  consequences  of  this  action 
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Abstract 

Working  with  models  and  simulations  has  a  long  tradition  in  the  building  industry.  The 
models  are  used  for  visualising  the  drawings  of  the  architect  and  to  discuss  possible 
problems  during  construction. 

Due  to  the  introduction  of  CAD-Systems  visualisation  has  been  expanded  potentially. 

The  geometry  of  an  architectural  project  is  digitised  to  create  a  three  dimensional  model 
within  the  computer.  Virtual  Reality  and  it’s  possibility  to  compute  3D-data  in  real  time, 
interactive  control  mechanisms  and  free  movability  inside  Cyberspace  opens  new  per¬ 
spectives  in  architectural  modelling. 

By  using  the  techniques  of  virtual  reality  it  is  possible  to  document  different  stages  of  a 
buildings  history,  enabling  the  viewer  to  interactively  walk  through  time.  This  could  be 
used  for  research  and  reconstruction  of  historic  buildings,  with  the  possibility  to  show 
different  stages  of  completion  and  changes  through  time. 

"House  Moven"  in  the  Westfalian  Outdoor  Museum  at  Detmold  has  been  modelled  in 
this  way  and  is  to  show  the  potential  of  virtual  reality.  It  is  possible  to  walk  through  the 
three  dimensional  model  and  to  pick  out  certain  aspects  of  construction  to  analyse  more 
closely. 

Furthermore,  changes  throughout  different  historic  periods  and  building  stages  since  it’s 
construction  in  the  16th  century  are  made  visible.  The  analysis  of  the  user  interface  be¬ 
tween  man  and  machine  which  is  to  be  used  by  academically  trained  and  by  people  with 
little  or  no  knowledge  of  computer  modelling  is  of  great  interest. 

Introduction 

The  task  of  combining  two  veiy  different  disciplines  has  a  particular  attraction  for  an 
engineer.  On  the  one  hand  there  is  the  preservation  and  documentation  of  historic 
monuments  -  a  very  old  discipline  with  roots  in  Historicism  and  in  Enlightenment  (Ger- 
mann  1980).  Virmal  Reality  or  Cyberspace  on  the  other  hand  are  rather  new  terms.  Their 
exact  meaning  has  not  yet  been  fixed.  Cyberspace  is  a  word  that  was  first  used  in  Science 
Fiction  Literature  (Gibson  1987).  The  objective  of  combining  those  disciplines  is  to  look 
for  a  man  computer  interface  that  facilitates  research  of  building  historians  by  allowing  a 
simple  interactive  and  immersive  navigation  through  historic  buildings. 
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Documentation  of  historic  monuments 

The  way  generations  of  historians  documented  their  work  is  measuring  the  building  with 
the  conventional  measuring  methods  of  the  surveyors  (Cramer  1993)  and  making  draw¬ 
ings  of  them  -  usually  drawn  to  a  scale  of  1:  20,  1:50  or  1:100  and  in  four  different 
grades  of  precision  (GebeBler  1986),  Results  of  research  were  marked  with  different  col¬ 
ours. 


Figure  1:  Manually  Drawn  Plan  of  House  Moven 

New  techniques  of  measuring  like  for  instance  photogrammetry  were  naturally  adopted. 
As  they  supplied  three  dimensional  data  in  x-y-z  co-ordinates  it  was  obvious  to  use  them 
for  building  a  three  dimensional  computer  model. 
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Figure  2:  Detail  of  Manually  Drawn  Plan 


There  are  excellent  examples  to  be  visited  in  the  Internet  as  for  instance  the  Amiens  Ca¬ 
thedral  Media  Centre  For  Art  History  of  the  Columbia  University  in  New  York  (Colum¬ 
bia  1997).  Another  excellent  example  of  the  possibilities  is  the  reconstruction  of  the 
Archaeological  Park  -  Xanten  by  the  "Instimt  fur  Numerische  Methoden  und  Morma- 
tionsverarbeitung"  -  University  of  Dortmund  (Rank  1995) 

Virtual  Reality 

The  term  "Virtual  Reality"  often  seems  to  be  woolly  and  vague.  Even  a  dictionary  is  not 
very  helpful,  as  virtual  there  is  defined  as  "almost"  ,  "nearly"  or  "seeming".  So  I  will  try 
to  give  an  example. 

Man's  left  and  right  eye  have  different  angles  of  view  so  they  produce  two  different 
"pictures".  Our  brain  is  able  to  blend  two  2-dimensional  pictures  into  one  3-dimensional 
image  of  our  surroundings.  What  we  can  see  is  a  3-dimensional  image  of  the  space  - 
there  brain  forms  a  "virtual  space".  This  virtual  space  is  shaped  by  our  natural  experi¬ 
ence.  Our  brain  adds  the  information  of  our  sense  of  touch,  taste  and  hearing  extending 
our  image  of  the  space.  Since  we  are  bom  we  are  living  in  a  virtual  space. 

Our  image  of  space  seems  natural  to  us  because  we  are  living  in  it  from  birth.  All  our 
experience  is  based  on  the  information  this  image  of  space  provides.  The  painting  of 
Ren6  Magritte,  the  surrealistic  painter,  entitled  "c’  est  ne  pas  une  pipe"  shows  us  the  di¬ 
lemma  of  our  mind:  What  we  see  on  the  painting  is  a  pipe  but  the  title  tells  us  it’s  only 
the  image  of  a  item  of  practical  use  as  the  artist  saw  it.  And  -  to  bring  this  idea  to  an  end 
-  the  pipe  I  am  smoking  in  the  evening  is  also  a  virtual  image  of  a  pipe  that  is  created  by 
my  sense  of  sight,  taste  and  touch. 

The  term  "Virtual  Space"  or  "Virtual  Reality"  changes  it’s  meaning  when  used  in  context 
with  computers.  It  describes  a  new  way  to  create  a  man  -  computer  interface.  In  this 
context  interface  means  all  units  that  let  man  manipulate  the  computer. 
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The  object  of  research  on  virtual  reality  is  to  improve  man  -  computer  interface  even 
more.  Many  problems  and  tasks  we  have  to  solve  with  computers  are  located  in  the  3- 
dimensional  space.  An  architect  for  example  wants  to  show  the  planned  building  to  his 
client,  so  that  he  can  judge  proportions,  light  and  materials. 

Virtual  Reality  with  computers  means  the  simultaneous  existence  of 

3D-input  and  -output, 
real  time  behaviour, 
autonomous  objects  and 
immersion. 

More  detailed: 

3D-input  and  -output  means  that  the  representation  as  well  as  the  sensors  of  the 
VR  system  are  designed  for  three  dimensions.  Suitable  equipment  is  for  in¬ 
stance  a  data  glove,  space  mouse,  head  mounted  displays  and  tracker,  stereo 
glasses. 

Real  time  behaviour  means  that  the  VR-System  reacts  on  user’s  input  or  move¬ 
ment  with  a  delay  less  than  10  to  50  milliseconds. 

Autonomous  objects  are  characterised  by  owning  properties  that  are  independent 
of  the  current  reaction  of  the  user  and  by  the  possibility  to  interact  with  them. 
Immersion  is  the  effect  that  the  user  is  mentally  completely  immersed  in  the  vir¬ 
tual  world. 

The  Task 

A  historic  building  is  rarely  found  in  the  state  it  was  built  centuries  ago.  The  buildings 
we  consider  have  changed  their  use  several  times.  They  have  partly  been  broken  down 
and  extended  in  other  parts.  With  changing  fashion  they  got  new  facades  -  up  to  seven 
times  in  six  centuries.  Therefore  it  is  extremely  difficult  to  decide  which  historic  state  of 
the  building  is  to  be  reconstructed.  The  earlier  and  later  states  of  this  building  can  only 
be  documented.  There  seems  to  be  a  need  for  a  system  that  supports  the  decision  by  visu¬ 
alising  the  different  states  of  the  building  in  an  easily  understood  way.  We  are  looking 
for  an  economic  solution  for  this  problem  as  a  compromise  between  costs  and  perform¬ 
ance.  We  suppose  that  Virtual  Reality  can  serve  as  a  tool  do  visualise  different  stages  of  a 
building  and  can  be  used  even  by  inexperienced  computer  users.  The  aims  of  the  project 
are: 


•  to  find  a  way  to  document  and  inventarize  the  different  states  of  a  historical 
building  for  research  in  a  3 -dimensional  way, 

•  to  make  visible  the  different  states  of  a  building  to  visitors  of  the  museum 
that  are  not  experienced  in  using  computers, 

•  to  test  different  man  -  computer  interfaces  for  their  suitability 
The  Tools 

As  we  had  to  regard  the  economic  efficiency  we  chose  a  PC-based  System  as  a  worksta¬ 
tion: 

•  Dual  processor  Pentium  200  MHz, 

•  256  MB  memory, 

•  4  GB  harddisk. 
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•  Diamond  Fire  GL  1000  SMB  graphic  card,  Diamond  Monster  3Dfx  graphics 
accelerator, 

•  Windows  NT  4.0  operating  system. 

Space  mouse  and  stereo  glasses  will  complete  the  '’virtual”  equipment. 

Although  all  used  to  design  with  the  CAD  system  MicroStation  and  Triforma  we  decided 
to  do  all  construction  work  with  3D  Studio  MAX  by  Kinetix.  Mainly  designed  as  a  pro¬ 
gram  for  visualisation  and  animation  it  has  the  capability  to  export  the  animated  model 
to  VRML  2.0,  the  common  description  language  for  Virtual  Reality.  Where  necessary  we 
improved  and  optimised  the  VR  model  with  the  help  of  the  program  Realimation  by 
Datapath. 

Users  can  immerse  in  the  Model  using  the  viewers  of  Realimation  or  Superscape  even 
within  the  WWW.  For  this  purpose  one  will  to  have  to  install  the  Cosmo  Player  plug-in 
for  Netscape  Navigator  by  Silicon  Graphics  or  the  World  View  plug-in  for  Netscape 
Navigator  by  Intervista. 


Figure  3:  Virtual  View  of  House  Moven 


An  Example 

House  Moven  is  part  of  the  "Paderbom  Village”  in  the  Westfalian  Outdoor  Museum  at 
Detmold.  hi  1980  it  was  pulled  down  at  it’s  original  site  in  Bruchhausen  and  rebuilt  1982 
to  1986  in  Detmold.  It  is  a  typical  half  timbered  construction  of  this  region.  House 
Moven  was  built  in  1651  by  Peter  Moven,  a  farmer.  He  used  the  foundations  of  a  former 
building. 

The  original  ground  plan  of  1651  is  to  make  the  history  easier  to  understand. 
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Grouncf  Floor  UpK^oor 


Figure  4:  Ground-plan  1651 


First  changes  were  made  in  the  18th  century: 

The  original  cellar  was  placed  below  the  living  -  room.  Owing  to  frequent  flooding  by  the 
silver  stream,  it  was  given  up  about  1 800.  In  place  of  the  old  one  a  new  cellar  was  built 
below  the  back  chamber.  It  could  be  accessed  from  the  kitchen  next  to  it.  As  the  fireplace 
was  exactly  where  the  staircase  to  the  cellar  was  to  be  built,  the  fireplace  was  moved  to 
the  opposite  comer. 


Figure  5:  Virtual  View  into  the  Cellar  of  the  18th  Century 


In  the  mid- 18th-century  the  value  of  the  house  was  increased  by  replacing  the  old  roof 
thatched  with  straw  by  a  roof  of  sandstone  slabs.  Additional  rafters  were  built  in  order  to 
take  the  additional  weight  of  the  stone  slabs. 
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Figure  6:  Roof  with  additional  rafters 

Substantial  changes  took  place  in  the  third  phase  of  rebuilding  in  the  19th  century. 

The  filling  of  the  gable  that  originally  was  made  of  straw  and  loam  was  replaced  by 
wooden  boards.  The  inhabitants  obviously  wanted  to  reduce  maintenance  of  the  house. 
The  living  room  was  expanded  by  building  a  bedroom  on  the  ground  floor  behind  the 
kitchen.  A  half  timbered  wall  was  built  to  separate  the  kitchen  from  the  hall.  As  smoke 
from  the  fireplace  could  no  longer  pass  through  the  hall  and  the  roof,  a  chimney  had  to 
be  built.  At  the  back  gable  an  extension  for  a  cowshed  was  added  to  the  house.  The  place 
won  in  the  old  cowshed  was  used  to  install  a  staircase  instead  of  the  old  ladder  to  the 
grain  loft. 

All  those  changes  were  reconstructed  out  of  the  observations  made  when  taking  down 
the  house.  The  other  sources  were  old  documents,  letters  and  bills.  They  are  documented 
in  drawings  and  sketches  true  to  scale  but  without  dimensions. 

Working  Method 

In  a  first  step  the  existing  building  that  was  reconstructed  in  the  state  of  1 860  was  redes¬ 
igned  using  3D  Studio  MAX.  Dimensions  were  taken  from  the  documentation  of  the 
museum  and  whenever  there  was  a  contradiction  they  were  taken  in  place. 

The  textures  of  wood,  loam,  stone  and  the  other  material  were  gathered  from  digitised 
photographs.  They  were  rectified  and  colour  corrected.  The  reachable  speed  of  movement 
in  the  VR  model  strongly  depends  on  the  size  of  the  texture  files.  So  rather  a  big  effort 
had  to  be  made  to  process  the  textures.  Finally  they  were  stored  in  JPEG  format  and 
linked  to  the  geometry  to  get  a  realistic  impression.  The  figures  in  this  paper  were  gener¬ 
ated  synthetically  by  means  of  this  3D-model. 


Figure  7:  Texture  of  a  Board  and  Geometry  of  the  Roof 

The  reconstruction  of  the  other  stages  required  intensive  discussion  with  the  building 
historians  because  their  findings  when  taking  down  the  house  needed  interpretation.  This 
interpretation  was  facilitated  as  designer  and  historian  could  refer  to  the  VR  model. 
Contradictions  in  the  conventional  plans  could  be  resolved  by  consulting  the  3D-model 
too. 

Elements  that  were  lost  during  the  centuries  like  for  instance  windows  were  replaced  by 
elements  of  other  houses  which  are  typical  for  the  period  to  be  shown. 

Results 

We  succeeded  in  building  VR  models  of  the  most  important  stages  of  house  Moven.  One 
can  walk  through  the  house,  look  at  details,  climb  up  to  the  upper  floor,  leave  the  house 
and  walk  around,  using  mouse  and  screen.  The  model  was  transferred  to  a  SGI  equipped 
with  space  mouse  and  shutter  glasses  and  provided  a  rather  realistic  impression. 

Links  in  the  VR  model  lead  to  explaining  text  about  the  history  of  the  house  and  it’s  in¬ 
habitants,  They  also  lead  to  animations  of  important  details.  As  other  links  also  lead  back 
to  the  VR  model,  all  the  information  is  connected  to  a  Hypermedia  document. 

Conclusions 

The  example  of  house  Moven  proves  that  it  is  possible  to  reproduce  an  old  half  timbered 
house  and  his  different  stages  through  the  time  as  a  VR  model.  Even  if  the  workflow  for 
producing  the  model  should  be  improved  for  further  projects,  the  described  method  of 
working  is  a  suitable  way  to  generate  a  VR  model  out  of  conventional  documents. 

With  about  70  000  polygons  in  the  model  the  PC  based  workstation  has  reached  it’s  limit 
so  it  is  not  yet  possible  to  reach  a  grade  of  precision  that  allows  to  consider  the  deforma¬ 
tions  of  a  building.  Whereas  a  PC-based  workstation  is  sufficient  for  designing  the  VR 
model  and  doing  historical  research,  one  should  consider  to  use  faster  computers  for 
presentation. 
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Abstract 

We  describe  a  series  of  related  research  projects  investigating  the  adoption  of  Internet 
technologies  in  a  small  town.  In  one  project  we  used  an  ethnographic  framework  to 
understand  communication  patterns  in  the  town  and  how  knowledge  about  and  use  of  the 
Internet  was  disseminated.  The  ethnographic  project  set  the  stage  for  two  other  projects 
we  did  in  the  same  town.  Here,  we  reflect  on  the  advantages  of  conducting  long-term 
qualitative  research  in  the  area  of  communications  technologies,  how  the  ethnographic 
project  informed  and  influenced  the  shape  and  findings  from  the  two  other  projects. 

Introduction 

The  Winona  Projects  were  undertaken  by  U  S  WEST  Advanced  Technologies  in  an  effort 
to  understand  how  the  technology  of  the  Internet  disseminates  into  a  community,  to 
understand  what  factors  and  what  technologies  facilitate  and  impede  Internet  adoption  by 
individuals,  families,  schools,  businesses,  governments,  and  other  institutions.  We  also 
hoped  to  cultivate  a  well-understood  testbed  for  numerous  products  and  services  that  we 
imagined  would  be  delivered  by  way  of  the  Internet  in  the  coming  years.  The  projects 
spanned  a  period  of  18  months  during  1995  and  1996.  They  encompassed  several 
different  research  and  development  efforts,  three  of  which  are  discussed  in  this  paper:  1) 
Networked  Community  Ethnographic  Project;  2)  Virtual  School  Project;  and  3) 
Communications  Rich  Web  Site  Project. 

The  three  projects  can  be  seen  on  a  continuum  of  complexity  and  level  of  detail.  The 
Networked  Community  Ethnographic  Project  was  an  ethnographic  study  of  technology 
diffusion  in  the  town  of  Winona,  providing  insights  into  the  complex  organizational  and 
individual  factors  contributing  to  the  genesis  of  an  on-line  community.  The  Virtual 
School  Project  focused  on  the  use  of  the  Internet  by  a  school  and  its  families.  The 
Communications  Rich  Web  Site  project  introduced  Internet  collaboration  technology  into 
a  maturing  Web  environment  for  use  by  community  leaders  in  town  meetings. 

We  discuss  both  the  findings  from  the  research  we  did  in  Winona  and  the  importance  of 
the  ethnographic  component  of  the  work.  The  ethnographic  project,  which  was  first, 
informed  and  influenced  focus,  data  analysis,  and  evaluation  of  the  other  projects  we 
carried  out  in  Winona.  The  data  collected  in  the  Virtual  School  and  Communications 
Rich  Web  Site  studies  revealed  their  full  meaning  only  in  the  context  of  our  involvement 
with  the  community  as  a  whole. 

Identification  of  early  adopter  individuals  using  traditional  marketing  criteria,  such  as 
demographic  and  consumer  characteristics  in  an  organization  did  not  adequately  predict 
adoption  of  the  technology.  While  individual  early  adopters  played  key  roles  in  Winona, 
it  was  critical  to  understand  how  Winona’s  various  social  institutions  supported  and 
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influenced  each  other.  An  assumed  value  of  the  Internet  was  the  access  it  gives  to  non¬ 
local  information  resources.  We  were  surprised  to  find  that  Winonans  were  drawn  to  the 
Internet  initially  for  its  ability  to  facilitate  communication  of  local  information  within  the 
community,  and  secondarily  for  access  to  remote  resources,  commerce,  and  long-distance 
communication.  We  hold  that  the  study  of  conamunication  technologies,  like  the  Internet, 
necessitates  an  approach  that  takes  social  complexity  into  account.  Using  an 
ethnographic,  whole-community,  approach  provided  us  with  a  framework  for 
understanding  what  and  who  was  communicated  with,  and  how  adoption  proceeded,  and 
would  be  likely  to  proceed  under  similar  circumstances. 

The  Winona  projects  and  their  interrelations  are  discussed  in  greater  detail  below.  We 
begin  with  a  discussion  of  the  history  of  the  projects  before  launching  into  each  project 
individually. 

The  Winona  Projects 
Background:  Why  Winona? 

“Breaking  Ground”  is  an  apt  metaphor  for  what  happened  in  Winona,  Minnesota,  when, 
in  1994,  U  S  WEST  supported  a  grass-roots  initiative  to  install  a  fiber-optic  network 
throughout  the  town  of  Winona,  Minnesota,  linking  eight  of  the  town's  most  prominent 
social  institutions.  The  initiative  began  when  a  prominent  business  leader  offered  a  seed 
grant  to  put  Winona  on  the  Internet,  the  “information  superhighway.”  Years  before, 
Winona  had  been  passed  over  for  a  place  on  an  Interstate  highway,  and  they  wanted  to 
make  sure  that  they  did  not  miss  another  opportunity  to  enhance  their  economic  base. 

Early  in  1995,  after  the  fiber  installation  had  been  completed  by  U  S  WEST,  but  before 
WWW/Intemet  service  was  commercially  and  locally  available  in  the  town,  we  learned 
about  Winona,  We  saw  this  as  an  opportunity  to  watch  adoption  of  a  new  technology 
from  the  ground  up. 

Winona  is  a  small  town  with  approximately  25,000  residents.  It  is  located  about  two-and- 
a-half  hours  south  of  Minneapolis  on  the  Mississippi  River.  We  made  our  first  foray  into 
our  new  fieldsite  two  months  prior  to  widespread  availability  of  the  Internet  in  Winona. 

In  May  of  1995,  Luminet,  a  local  Internet  Service  Provider  (ISP),  began  offering  service 
to  residents,  businesses  and  other  organizations.  Within  one  year,  about  ten  percent  of 
private  households  had  gained  Internet  access\  and  community  organizations  and 
businesses  had  embraced  the  WWW  as  a  new  publication  and  communication  medium 
[1]. 

In  the  fall  of  1995  an  educational  initiative  called  The  Virtual  SchooF^  (TVS)  began, 
bringing  240  families  from  a  private  high  school  on-line  [2].  TVS  now  links  more  than 
500  families  whose  children  attend  parochial  schools  in  Winona.  Two  goals  of  the  TVS 
initiative  were  to  make  communication  between  parents  and  school  staff  easier,  and  to 
free  the  educational  enterprise  from  the  constraints  of  the  school  building  and  extend  it 
into  homes  and  the  broader  community. 


^  This  number  only  represents  subscribers  to  Luminet;  it  does  not  include  those  who  had  Internet  access 
through  an  on-line  service,  one  of  the  universities,  K-12  educational  institutions,  work,  or  public 
computing  facilities. 


381 


NETWORKED  COMMUNITY  ETHNOGRAPHIC  PROJECT 
Breaking  New  Ground 

Although  ethnographic  techniques  have  been  used  in  the  field  of  human  computer 
interaction  (HCI)  since  the  1980s  [3,  4,  5],  only  a  few  HCI  researchers  have  attempted,  or 
seen  a  need  for  conducting  long-term  ethnographic  research  of  a  more  anthropological 
nature  in  complex  communities  comprised  of  multiple,  interrelated,  and  interacting  social 
organizations.  The  Networked  Community  Ethnographic  Project  was  such  an  effort. 

The  Networked  Community  Ethnographic  Project  set  the  stage  for  the  other  Winona 
Projects.  Initially,  we  wanted  to  understand  how  information  about  the  Internet  is 
disseminated  in  the  context  of  a  town,  what  beliefs,  if  any,  people  in  Winona  held  about 
the  Internet,  and  how  the  use  of  the  technology  itself  would  proceed.  We  were  interested 
in  understanding  this  at  both  the  organizational  and  individual  levels. 

One  of  the  concerns  we  had  in  doing  our  research  in  Winona  was  its  representativeness. 
After  all,  it  is  a  small  rural  town  that  is  relatively  homogenous  demographically 
compared  with  many  of  the  more  urban  areas  served  by  our  company.  Although  there 
were  no  equivalent  smdies  with  which  we  could  compare  our  findings,  we  kept  a  close 
watch  on  the  various  data  collection  activities  surrounding  the  Blacksburg  Electronic 
Village  (BEV)[6],  the  HomeNet  Project  [7],  and  the  Boulder  VaUey  Internet  Project  [8, 
9].  This  helped  us  discern  idiosyncratic  findings  from  those  findings  that  were  more 
generalizeable.  With  few  exceptions,  our  findings  in  Winona  were  consistent  with  what 
researchers  in  these  other  contexts  were  finding. 

At  the  outset,  we  felt  that  if  we  were  later  going  to  be  able  to  understand  any  changes  in 
communication  patterns  related  to  the  Internet,  we  needed  to  first  understand  modes  of 
communication  used  in  the  town  without  it.  To  this  end,  we  documented  public  forms  of 
communication  of  local  information,  including  bulletin  boards,  flyers,  newspapers,  radio, 
television,  and  so  forth.  We  documented  the  form  of  the  communication,  the  content,  the 
location,  and  the  intended  audience  for  the  information  (e.g,,  students,  women,  tourists, 
etc.).  This  data  collecting  activity  was  fruitful,  because  it  gave  us  a  good  overview  of  the 
town  and  the  issues  and  events  that  were  of  public  importance.  The  two  local  newspapers 
were  invaluable.  From  them  we  learned  critical  political  information,  as  well  as  received 
an  introduction  to  a  cast  of  characters  with  whom  we  would  later  form  important 
relationships.  We  knew  in  advance  where  to  tread  lightly,  and  where  we  would  likely  be 
welcomed  with  open  arms.  Information  we  collected  during  this  phase  also  gave  us 
plenty  of  conversational  content,  so  that  when  we  were  talking  with  Winonans  we  could 
understand  tangential  references  and  subtle  nuances  of  what  people  were  saying. 

During  the  early  phase  of  the  ethnographic  study  we  wanted  to  learn  how  aware  of  the 
Internet  people  in  Winona  were:  if  they  had  heard  of  it,  if  they  had  used  it,  if  they  had  any 
beliefs  about  it  or  technology  in  general.  To  collect  data  on  this  we  informally 
“interviewed”  people  we  met  on  the  street,  people  we  met  in  restaurants  and  bars,  shop 
owners,  and  whatever  Winonan  crossed  our  path.  These  interviews  were  more  like 
directed  conversations  than  interviews.  Typically  they  would  begin  as  any  conversation 
between  strangers,  with  an  exchange  of  pleasantries  about  the  weather,  a  conmient  on 
how  nice  the  town  was,  leading  into  the  fact  that  we  were  visiting,  working  for  U  S 
WEST  on  a  project  having  to  do  with  the  Internet.  From  here,  the  conversation  would 
diverge  in  many  directions  depending  on  the  interests  of  the  person  we  were  talking  to. 
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When  we  first  arrived  in  Winona  in  the  Spring  of  1995,  awareness  of  the  Internet  was 
limited.  The  majority  of  the  people  we  spoke  with  had  heard  of  it  through  local  media, 
but  had  never  used  it  in  any  way.  For  most  people  the  Internet,  the  World  Wide  Web,  and 
Luminet,  the  local  ISP,  were  synonymous.  A  small  percentage  of  Winona  households 
subscribed  to  on-line  services  (AOL,  CompuServe).  Most  of  those  people  who  had  used 
the  Internet  to  send  email  or  had  used  an  on-line  service  had  never  seen  the  World  Wide 
Web.  There  were  already  a  couple  of  Web  sites  hosted  in  the  town.  The  most  extensive 
site  was  at  St.  Mary’s  University  of  Minnesota  (then  St,  Mary’s  College).  Winona  State 
University  did  not  have  a  web  site,  but  they  were  using  the  email  capacities  of  the 
Internet,  and  were  heavily  involved  in  the  Luminet  initiative.  They  also  hosted  a  gopher 
server  for  the  campus.  In  both  universities,  Internet  access  was  Hmited  primarily  to 
faculty,  but  both  schools  had  plans  for  expansion  to  their  entire  student  bodies,  as  well  as 
expansion  of  their  networks  into  the  residential  halls.  Only  two  larger  businesses  in 
Winona  had  Web  sites,  but  these  were  hosted  in  large  metropolitan  areas  outside  of 
Winona.  TTiere  was  also  a  private  not-for-profit  technology  company,  Vanguard 
Technology  Group,  that  had  just  begun  selling  its  services  for  Web  hosting  and  page 
development,  but  initially  their  only  client  was  Cotter  High  School  (discussed  in  detail 
below). 

As  the  Luminet  launch  date  loomed  nearer,  public  awareness  of  the  Internet  grew.  The 
newspapers  ran  articles  almost  daily  on  Luminet  developments  and  activities  surrounding 
it.  The  radio  stations  publicized  these  events  as  well.  At  this  point,  we  concentrated  our 
efforts  on  understanding  what  visions,  if  any,  people  in  the  town  had  for  the  technology, 
and  whether  they  were  acting  on  their  visions  in  any  way.  As  the  technology  matured,  we 
tracked  developments  so  that  we  might  gain  a  better  understanding  of  barriers  and 
catalysts  of  adoption.  We  formally  interviewed  individuals  in  all  of  the  major  social 
institutions  with  direct  drops  into  the  network  (hospital,  city  hall,  Winona  Technical 
College,  Winona  State  University,  Winona  Middle  School,  Winona  High  School,  Cotter 
High  School)  as  well  as  individuals  from  social  institutions  without  connections  (e.g.,  the 
chamber  of  commerce,  public  library,  and  the  senior  center),  proprietors  of  small 
businesses,  and  employees  of  medium,  and  larger  businesses. 

Adoption  of  the  Internet 

We  learned  that  having  direct  connectivity  to  the  fiber-optic  network  did  not  necessarily 
predispose  organizational  adoption  of  the  technology,  nor  did  technical  knowledge.  For 
example,  in  the  months  preceding  the  launch  of  Luminet,  there  was  much  talk  about  the 
benefits  of  the  network  to  the  social  institutions  with  direct  drops  to  the  network. 
Everyone  assumed  that  these  institutions  would  be  the  first  to  adopt  the  technology  and  to 
use  it  in  innovative  ways.  This  did  not  turn  out  to  be  the  case.  For  example,  numerous 
articles  were  published  in  the  local  papers  on  how  telemedicine  would  enhance  the 
hospital’s  ability  to  serve  the  health  needs  of  rural  patients;  the  hospital  would  have  a 
direct  link  to  the  Mayo  Clinic  in  neighboring  Rochester,  using  the  two-way  video 
capabilities  provided  by  Luminet.  Other  visions  included  on-line  patient  registration,  and 
consultations.  Today,  these  visions  have  still  not  been  realized.  When  we  interviewed 
various  stakeholders  at  the  hospital,  we  discovered  these  visions  emanated  from  a  single 
technology  champion.  The  hospital  administrator  and  systems  support  staff,  however, 
were  skeptical  about  the  Web  and  the  role  it  would  play  for  them.  Lack  of  security  was  a 
concern  as  well  as  the  expense  of  equipment  that  would  be  required  to  make  these  visions 
a  reality. 
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On  the  other  hand,  at  the  Winona  Senior  Friendship  Center  (a  city  department  without 
direct  connectivity),  the  director  and  a  senior  volunteer  had  more  modest  visions  of  what 
the  Internet  could  do  for  the  aging  population  of  Winona.  They  envisioned  a  scenario 
where  senior  citizens  would  serve  as  mentors  to  young  people  over  the  Internet.  The 
director  of  the  center  did  not  need  previous  technology  exposure  to  understand  the  great 
value  that  the  Internet  could  bring  to  the  seniors  in  her  center  ,  She  collaborated  with  a 
volunteer  at  the  center  who  had  more  technical  experience.  Together,  they  went  to  local 
businesses  to  raise  money  for  their  computer  lab.  Local  businesses  immediately  perceived 
the  value  of  the  project  and  were  willing  to  donate  to  the  cause.  Also,  because  the  senior 
center  fell  under  the  auspices  of  the  city  government.  City  Hall’s  support  was  a  critical 
factor  in  making  the  Sr.  Friendship  Center  computer  lab  a  reality.  They  installed  a  small 
computer  lab  with  dial-up  Luminet  service,  and  had  made  arrangements  for  members  of 
the  senior  center  to  get  free  email  through  a  state  university.  Seniors  literally  lined  up 
outside  the  door  before  the  center  opened  to  claim  their  spot  at  a  terminal,  and 
membership  at  the  center  doubled  in  a  short  period.  Visionaries  and  champions  of  the 
technologies  emerged  from  unlikely  places. 

What  happened  at  the  Sr.  Friendship  Center  illustrates  that  adoption  of  the  Internet  not 
only  occurs  in  technologically  prepared  organizations,  but  also  in  places  where  individual 
visions  are  met  with  the  support  of  the  community.  We  quickly  understood  that  adoption 
of  the  technology  was  not  going  to  occur  in  institutions  that  lacked  individual  champions 
of  the  technology,  or  institutions  where  political  and  bureaucratic  barriers  were  difficult 
to  overcome. 

Another  institution  that  was  critical  to  raising  Internet  awareness  was  the  Winona  Daily 
News,  which  launched  an  experimental  “on-line  newspaper”  linking  to  a  plethora  of 
community  and  regional  information.  City  Hall,  and  some  of  its  associated  organizations 
also  played  an  important  role  in  fostering  Internet  adoption.  Although  City  Hall  had 
expansive  visions  of  how  to  use  the  Internet  to  facilitate  community  involvement  in  city 
government,  it  has  not  been  able  to  bring  its  visions  to  light  yet.  The  city  manager,  and  a 
number  of  other  city  officials  constantly  championed  technology  efforts  in  the  town.  The 
visions  and  accomplishments  of  both  the  newspaper  and  City  Hall  are  described  in 
greater  detail  in  our  discussion  of  the  Communications  Rich  Web  Site  below. 

It  was  the  synergy  of  efforts  in  three  main  sectors-the  public,  businesses,  and  the 
educational  sector-that  turned  Winona  into  an  active  on-line  community  during  the 
groundbreaking  phase.  Businesses  expected  revenue  and  therefore  invested  not  only  in 
their  own  systems  but  provided  financial  help  to  public  institutions.  Schools 
disseminated  access  to  the  Internet  into  the  families  and  thus  created  an  audience  and 
active  participants. 

THE  VIRTUAL  SCHOOL  PROJECT 
Schools  as  Sources  of  Technology  Diffusion 

The  Virtual  School  Project  was  a  natural  extension  of  the  ethnographic  study  of  the  town, 
and  indeed  overlapped.  The  goal  of  the  Virtual  School  Project  was  to  understand  Internet 
adoption  at  a  more  detailed  level;  to  focus  on  a  single  organization~the  school-and,  also 
to  understand  some  of  the  dynamics  of  Internet  adoption  at  the  family  level.  From  the 
ethnographic  research  that  preceded  the  Virtual  School  Project  we  knew  that  schools  in 
Winona  were  key  points  of  dissemination  of  information  about  the  Internet,  and  also 
provided  Winonans  with  their  first  hands-on  experiences  with  the  WWW.  This  was  also 
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apparent  nationally;  there  had  been  an  almost  exponential  growth  of  school  entries  in  the 
on-line  school  registry  Web66,  and  Clinton’s  administration  had  issued  a  challenge  to 
businesses  to  provide  on-line  access  in  every  classroom  by  the  year  2000, 

In  Winona,  educators  in  the  public  school  system  showed  an  early  interest  in  the  Internet, 
mainly  because  it  promised  access  to  unlimited  teaching  materials.  The  main  concerns 
were  that  most  of  the  public  schools  had  outdated  systems,  and  many  teachers  felt  they 
lacked  the  preparation  and  skills  required  to  use  the  Internet.  This  is  not  unlike  the 
situation  at  many  public  schools,  at  that  time  [11].  Another  Winona  project,  not 
described  here,  provided  educators  at  the  public  middle  school  with  the  needed  resources. 

During  the  summer  of  1995,  while  we  continued  ethnographic  data  collection,  we 
established  a  good  working  relationship  with  individuals  at  the  Vanguard  Technology 
Group,  a  non-profit  business  that  was  founded  to  support  technology  efforts  at  the 
Catholic  Cotter  High  School.  Cotter  was  ahead  of  most  other  schools,  both  inside  and 
outside  of  Winona,  with  respect  to  the  use  of  technology.  Even  at  the  beginning  of  the 
study,  the  school  boasted  broadcast  cable  connectivity  into  every  classroom,  a  computer 
lab  with  30-1-  well-equipped  Apple  Macintoshes,  a  classroom  equipped  for  collaborative 
work  on  Apple  workstations,  ethemet  connectivity  across  campus  into  the  dorms,  and  its 
own  multimedia  production  lab,  run  by  Vanguard.  Clearly,  “access”  itself  was  a  non¬ 
issue  at  this  institution. 

Integrating  Learning  with  Community  Citizenship 

Vanguard’s  vision  for  using  the  new  town  network  was  more  far-reaching  than  what  had 
been  installed  into  the  school.  It  was  to  extend  Cotter’s  educational  reach  from  the 
campus  into  the  students’  homes,  and  thus,  into  the  community  by  providing  all  student 
families  with  in-home  Internet  connectivity.  The  educational  goal  was  to  contextualize 
education,  to  provide  access  to  educational  means  to  the  whole  community,  and  to  teach 
the  students  in  the  rich  context  of  that  community  instead  of  the  bounds  of  the  physical 
campus.  During  our  first  six  months  of  work  in  Winona,  this  project  was  bom  out  as  the 
‘Virtual  SchooP’^  Project”  (TVS), 

In  more  concrete  terms,  this  translated  into  the  goal  of  equipping  all  four-hundred 
families  of  Cotter  smdents  with  Macintosh  Performa’s'*'^  and  modems,  and  all  family 
members  with  individual  email  accounts.  The  families  would  also  have  access  to  the 
World  Wide  Web  through  Netscape'^^  browsers.  The  Vanguard  staff  had  started 
developing  the  TVS  Website  that  would  serve  as  the  “virtual  campus.”  These  pages 
included  a  school  calendar  published  weekly  with  daily  curricular  and  extra-curricular 
activities,  and  the  school  menu,  also  updated  weekly.  Faculty  and  student  directories 
were  planned,  with  pictures,  some  personal  information  and  email  addresses.  Teachers 
would  be  able  to  publish  assignments  and  class  materials  on-line,  and  students  could  post 
their  assignments  even  from  their  home  computers.  A  forum  would  provide  opportunity 
to  discuss  topics  of  interest  to  the  school  or  community  in  an  open  fashion.  Finally,  there 
was  an  area  for  “Hot  News”  where  school  cancellations,  or  announcements  of  PTA 
meetings  and  the  like  could  be  posted.  There  was  also  an  area  showing  exceptional 
student  work,  and  a  student  art  exhibition.  In  essence,  TVS  Web  site  was  planned  to  be 
the  communication  switchboard  for  Cotter  High  School,  see  Figure  1. 

During  the  first  year,  participation  in  this  trial  was  seen  as  voluntary;  however,  eventually 
in-home  connectivity  would  become  a  requisite  for  becoming  a  Cotter  student.  Parent 
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meetings  to  kick  off  the  trial  were  full  of  controversy:  Would  families  with  older 
computers,  and  especially  those  with  PC’s,  be  able  to  participate,  and  who  would  provide 
technical  support  for  those?  Was  the  lease  for  the  Performa’s  provided  by  the  school 
really  a  good  deal?  How  would  parents  protect  their  children  from  accessing  unwanted 
material,  and  would  this  program  eventually  discriminate  against  children  from  less 
affluent  households?  Finally,  was  the  school’s  heritage-to  provide  good,  basic  Catholic 
education  to  all  families  who  wanted  it-be  threatened  by  this  new  turn  towards 
technology?  However,  after  all  was  said  and  done,  about  250  families  (over  60  percent  of 
all  Cotter  families)  had  signed  up  to  be  members  of  the  trial  in  the  first  year  alone! 

The  research  project 

It  was  through  our  work  with  Vanguard,  during  the  Networked  Community  Ethnographic 
Project  phase  that  we  succeeded  at  recruiting  fifty  families  (155  individuals)  to 
participate  in  an  in-depth  study  of  Internet  adoption  and  use.  The  Virtual  School  Project, 
unlike  the  ethnographic  project,  relied  more  heavily  upon  quantitative  data  collection 
techniques,  but  we  used  qualitative  approaches  as  well.  This  project  focused  on  the 
experience,  attitudes,  and  use  of  the  Internet  of  fifty  families  as  they  came  on-line. 

Our  plan  was  to  roll  out  a  questionnaire,  tracking  family  members’  attitudes  toward 
technology,  their  interests,  self-reported  Internet  activities,  use  of  other  media,  and 
attitudes  toward  their  community  several  times  during  the  trial,  starting  a  few  weeks  after 
the  official  kick-off  in  the  fail  of  1995.  These  data  were  to  be  used  as  predictors  of  the 
on-line  tracking  data  we  planned  to  collect.  We  collected  data  on  email  activities,  as  well 
as  URL  accesses  of  each  individual  family  member.  Finally,  we  planned  to  do  in-home 
interviews  with  the  families  participating  in  our  study,  to  follow  up  on  any  interesting 
data  patterns  that  we  observed,  and  to  learn  more  about  in-home  dynamics  of  Internet 
adoption. 
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Figure  1:  Cotter's  Virtual  School  home  page.  Printed  with  permission  of  The  Virtual  SchooP'^  of  Winona 


Connecting  250 families 

To  facilitate  family  participation  in  our  study,  and  to  collect  observational  data  on 
parents’  first  experiences,  we  helped  out  in  eight  parent  training  sessions  held  by 
Vanguard.  In  these  sessions  parents  were  introduced  to  the  Internet,  the  use  of  email  and 
the  Webbrowser.  They  were  guided  through  everything  from  taking  their  equipment  out 
of  the  boxes  to  connecting  the  modem.  Modem  configurations  were  pre-loaded  on  all 
machines  that  were  leased  to  the  families.  A  few  months  into  the  trial,  however,  we  still 
had  not  recruited  many  families  to  participate  in  our  study.  Also,  email  accounts  had  not 
been  set  up  in  a  manner  that  allowed  us  to  track  individual  use;  we  could  only  track 
family  use.  We  finally  resorted  to  calling  the  250  families  by  phone  to  elicit 
participation,  and  found  that  a  large  majority  of  families  had  not  yet  established  a  reliable 
connection  to  the  Internet. 


Technical  problems  abounded,  and  while  family  members  had  explored  their  help  options 
(calling  the  local  ISP,  Luminet,  and  the  TVS  staff),  many  had  not  been  able  to  resolve 
their  difficulties.  Nobody,  including  ourselves,  had  been  able  to  estimate  the  amount  of 
technical  support  that  connecting  250  families  would  bring  along.  Server  outages  and 
other  operational  problems  added  to  the  confusion.  Help  material  and  hints  had  been  sent 
by  email  to  users  who  were  still  struggling  to  configure  their  modems,  but  because  they 
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hadn’t  been  able  to  configure  their  modems,  they  couldn’t  get  to  this  information. 
Phones  in  the  offices  rang  hot,  and  family  members  grew  frustrated,  because  they  could 
rarely  talk  to  somebody  and  work  at  the  problem  at  the  same  time;  problem  solving 
required  using  the  only  phone  line  in  many  households.  At  the  end  of  our  project,  about 
20%  of  the  families  had  ordered  second  phone  lines.  Finally,  many  of  the  families 
required  a  quick  visit  to  diagnose  and  fix  problems;  however  nobody  was  available 
during  evenings  and  weekends,  when  the  family  members  were  at  home  and  had  time  to 
work  on  their  computers. 

We  responded  by  hiring  a  local  college  student  who  provided  help  to  study  participants 
during  off-business  hours,  when  they  most  needed  the  help.  He  was  a  local  who  knew 
many  of  the  parents  and  understood  how  react  to  their  fear  of  the  new  family  appliance. 
Most  of  the  problems  resided  in  the  area  of  modem  configurations,  but  often  family 
members  just  needed  some  help  setting  up  passwords,  or  using  their  email.  In  June,  most 
families  had  recovered  from  the  initial  difficulties,  and  our  two  questionnaires  (one  sent 
out  at  the  very  end  of  1995,  and  the  second  in  May  1996)  documented  this  success. 

Problem  solvers 

Our  first  questionnaire  (answered  by  65%  of  the  study  participants,  the  second 
questionnaire  was  answered  by  46%)  showed  that  parents  and  teens  both  found  the  set-up 
tasks  of  connecting  the  modem  and  configuring  their  machines  relatively  difficult,  even 
though  the  teens,  especially  the  boys,  found  no  problems  with  the  actual  Internet  tasks: 
sending  and  receiving  e-mail  and  web-browsing.  Parents  admitted  to  having  some 
problems  with  both.  Here,  we  also  found  strong  indication,  that  overall,  the  teens  were 
more  self-sufficient  in  solving  problems  by  trial- and-error,  or  by  simply  relying  on  their 
social  network  for  support  [12], 

From  follow-up  interviews  we  learned  that  parents  usually  lacked  a  knowledgeable  social 
network  on  which  they  could  call  for  help.  We  had  anticipated  that  the  teens  would 
become  the  resident  experts,  but  often  they  could  not  solve  the  initial  configuration 
problems  either,  and  our  family  interviews  revealed  that  they  were  impatient  with  their 
parents’  questions  and  concerns,  leaving  the  parents  looking  for  help  in  other  places. 

Figure  2  shows  that  it  was  the  mothers  who  made  most  of  the  help  calls  during  the  first 
months,  even  though  mothers  expressed  the  greatest  degree  of  difficulty  with  the  new 
technology.  We  believe  that  mothers  took  on  this  responsibility  to  a  larger  degree  because 
only  53%  of  the  mothers  worked  full-time,  compared  to  96%  of  the  fathers,  thus  giving 
them  greater  access  to  the  home  and  computer  during  business  hours.  Figure  2  also  shows 
that  local  support  lines  were  called  most  often,  and  over  a  longer  time  period  than  help 
lines  of  far-off  computer,  software,  or  modem  manufacturers. 
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Figure  2:  Family  members’  reports  of  calling  5  help  lines  during  January  and  June 


Cruising  along 

By  June,  these  initial  difficulties  had  been  overcome  by  most  families,  so  that  their  and 
our  attention  was  shifted  to  the  Internet  content  and  function.  Unfortunately  for  us,  this 
marked  the  end  of  our  Internet  study,  but  in  the  last  few  months  we  had  been  able  to 
collect  some  information  about  what  seemed  valuable  and  intriguing  to  study 
participants. 

Email  enjoyed  great  popularity  with  almost  all  family  members;  however  an  analysis  of 
the  logged  email  activity  showed  that  parents  with  higher  levels  of  education  and  with 
office  jobs  were  more  active  than  parents  with  lower  levels  of  education  and  in  non-office 
professions.  In  the  interviews  we  found  out  that  for  the  teens,  checking  email  after  school 
had  become  a  new  ritual  (similar  to  checking  the  answering  machine).  The  parents  were 
checking  their  email  less  frequently;  however  they  did  express  great  excitement  about  re¬ 
connecting  with  old  friends  and  family  members  in  other  parts  of  the  United  States,  and 
even  in  other  parts  of  the  world.  They  also  found  email  a  welcome  medium  for 
communicating  with  teachers  who  were  otherwise  hard  to  reach,  for  keeping  in  touch 
with  their  older  college-bound  children,  and  even  for  maintaining  a  “conversation”  with 
children  living  with  them,  when  busy  schedules  had  made  meaningful  communication  a 
rarity. 

WWW  use,  logged  by  us,  varied  greatly  with  the  school  day,  and  with  the  school  year, 
suggesting  that  the  teens  were  the  actual  users  of  this  medium.  The  family  interviews 
supported  this:  parents  often  complained  that  the  children  were  occupying  die  computer 
during  the  evening  hours.  The  Virtual  School  pages  were  the  most  frequently  accessed 
pages  that  we  were  able  to  monitor.  Children  and  parents  alike  reported  using  the  pages 
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to  check  homework  assignments  and  school  schedules.  Parents  even  complained  that  the 
pages  were  not  updated  more  frequently  by  the  teachers  and  more  diligently  by  all  the 
teachers. 

Chat  rooms  were  especially  popular  with  the  teens.  A  cursory  content  analysis  conducted 
on  the  posts  to  the  chat  rooms  suggested  that  the  teens  were  actually  playing  a  virtual 
hide  and  seek  within  their  peer  group.  Several  Cotter  students  seemed  to  hang  out  in  a 
chat  room  and  give  off  clues  about  their  identities.  Strangers  were  of  little  interest 
compared  to  the  possibility  of  talking  to  somebody  well-known,  in  a  different  setting, 
under  different  rules  of  communication.  In  interviews,  many  of  teens  told  us  that  their 
siblings  were  using  chat  rooms  a  lot,  but  since  the  parents  were  always  present  in  these 
interviews,  we  never  had  a  chance  to  gain  more  insight  into  these  activities. 

On  the  whole,  the  Virtual  School  Project  provided  us  with  important  insights  into  how  the 
use  of  Internet  technology  would  diffuse  from  a  school  trial  into  the  homes  of  families. 
The  Winona  families,  in  spite  of  their  initial  doubts  about  this  trial,  were  remarkably 
willing  to  invest  in  what  they  believed  to  be  an  important  educational  tool  for  their 
children.  But  once  these  “educational  tools”  had  entered  the  households,  the  parents 
quickly  discovered  what  their  children  had  already  known  from  their  previous  experience 
with  the  networked  school  computers:  that  they  were  great  tools  for  social,  but  also 
professional  communication,  and  a  great  source  of  entertainment  along  with  the 
educational  application.  Even  though  many  parents  eventually  gave  up  on  the  use, 
parents  with  higher  levels  of  education  became  regular  fans  of  email,  and  some  of  them 
developed  an  interest  in  Web  surfing  as  well. 

Most  complaints  we  heard  were  in  the  form  of  wanting  to  have  more-better  access,  a 
second  phone  line,  faster  modems  and  more  bandwidth  (several  parents  inquired  about 
the  availability  of  ISDN),  and  even  more  information  at  the  Virtual  School  site.  Few,  if 
any,  families  entertained  thoughts  of  getting  rid  of  the  new  appliance,  despite  the 
technical  hassles  they  had  experienced. 

The  greatest  challenge  for  the  Virtual  School  project  on  the  whole  was  not  the  technical 
hurdle  of  initial  connections  but  the  necessity  of  involving  all  teachers  and  administrators 
at  Cotter  for  the  full  success  of  the  project.  The  communication  switchboard  could  only 
be  successful  if  the  end  nodes  (all  teachers  and  students)  were  active.  Not  all  Cotter 
teachers  were  enthusiastic  about  The  Virtual  School.  Some  of  them  felt  that  updating 
materials  and  the  requirement  of  posting  lesson  plans  was  burdensome,  and  time- 
consuming,  and  Internet  communications  with  the  parents  had  not  become  routine 
enough  to  help  reduce  time  spent  in  parent  meetings.  While  the  Virtual  School  Project 
was  successful  in  its  first  year,  some  parents  noticed  that  their  expectations  may  have 
been  unrealistic.  They  had  anticipated  even  more  information  and  more  widespread  use 
of  the  medium.  These  insights  gained  from  the  TVS  project  provide  another  piece  of 
evidence  for  our  argument  that  support  of  the  community,  in  this  case  the  Cotter 
community,  is  just  as  important  for  networked  technology  projects  as  access  to 
technology,  vision,  and  early  adopting  individuals. 

COMMUNICATIONS  RICH  WEB  SITE 

The  third  study,  the  Communications  Rich  Web  Site  project,  was  of  a  different  nature 
from  its  predecessors;  it  was  more  applied.  The  ethnographic  work  we  had  already  done 
in  Winona  set  the  stage. 
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The  goal  of  this  project  was  to  explore  how  leaders  in  the  community  responded  to  trials 
of  new  Internet  collaboration  technology  integrated  into  their  now  maturing  Intemet/Web 
environment.  We  envisioned  that  community  groups  would  use  communication-rich  Web 
sites  to  participate  in  real-time  forums  using  low  bandwidth  video  conferencing,  audio 
conference  calls,  shared  white  boards,  group  synchronized  tours  of  web  pages,  voting, 
and  text  chat.  They  would  also  have  tools  for  asynchronous  communication:  bulletin 
board  discussion  groups,  document  transfer,  and  electronic  mail.  We  wanted  to 
determine  whether  these  groups  would  get  some  of  the  same  benefits  that  businesses 
received  in  a  previous  study  of  desktop  conferencing  over  broadband  ATM  networks  that 
U  S  WEST  carried  out  in  the  State  of  Oregon  [13].  But  we  also  hoped  to  understand  how 
the  social  dynamics  of  the  conununity  influenced  the  design,  and  support  requirements 
for  the  services. 

Visions  of  Collaboration  -  Collecting  Requirements 

During  the  Networked  Community  Ethnographic  Project,  we  had  identified  a  variety  of 
public  and  private  communication  forums  in  the  community  that  played  important  roles 
in  education,  political  decision  making,  and  creating  and  maintaining  various  business 
coalitions.  And  we  had  tracked  the  role  that  the  two  local  newspapers  played  in 
communicating  the  local  educational,  political,  and  business  issues  as  well  as  attitudes 
toward  and  visions  for  use  of  the  Internet.  We  had  seen  how  the  World  Wide  Web  was 
augmenting  this  existing  communication  matrix  and  wanted  to  determine  how  real-time 
conferencing  tools  might  extend  this  communication. 

We  supplied  the  collaborative  tools  to  three  community  organizations  that  we  had 
interviewed  more  extensively:  the  Chamber  of  Commerce,  the  City  Council,  and  the 
Winona  Daily  News  on-line  newspaper  and  affiliated  Home  101  radio  station.  Leaders  in 
each  of  these  organizations  agreed  to  champion  on-line  meetings  in  which  they  would 
attempt  to  complete  their  ongoing  work. 

Chamber  of  Commerce 

We  had  gained  a  preliminary  understanding  of  some  of  the  groups’  needs  and  visions 
from  our  earlier  interviews.  The  Chamber  of  Commerce,  as  in  any  town,  supported  the 
needs  of  local  businesses.  An  important  goal  for  Winona  was  to  attract  both  local  tourists 
and  far  Eastern  tourists  who  found  a  sleepy  river  town  particularly  attractive.  They  had 
already  established  a  web  site  describing  Winona’s  value  as  a  tourist  destination.  But  they 
also  had  visions  of  improving  communication  at  other  levels.  They  envisioned  holding 
their  state-wide  meetings  with  other  Chambers  using  desktop  conferencing  to  avoid 
extensive  travel.  And  when  they  saw  a  demonstration  of  the  conferencing  technology 
integrated  into  a  web  site,  they  extended  their  vision  to  new  local  communications 
opportunities.  They  now  saw  the  web  as  an  opportunity  for  educating  local  employees 
that  served  the  tourist  populations  about  the  tourist  attractions  in  the  town  so  they  could 
refer  business.  They  also  saw  it  as  a  way  to  recruit  local  students  to  take  jobs  in  the  local 
tourist  industry. 

Winona  Daily  News 

The  local  Winona  Daily  newspaper  had  now  established  an  on-line  Web-based 
newspaper  that  was  attracting  significant  readership  as  well  as  business  fi*om  advertising. 
The  readership  was  both  local  and  geographically  distributed,  "Ex  Winonans"  checked 
the  site  to  stay  in  touch  with  what  was  going  on.  The  on-line  newspaper  staff  recognized 
that  there  were  new,  unexplored  opportunities  for  attracting  both  types  of  business  by 
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including  real-time  interactive  communication.  They  had  already  extended  the  traditional 
Letter  to  the  Editor  pages  to  on-line  Bulletin  Board  Forums.  Now  they  envisioned  live- 
interactive  competitions  in  activities  like  card  games  that  newspaper  readers  could 
observe  or  participate  in. 

Winona  City  Council 

The  City  Council  had  a  far-reaching  vision  of  the  role  of  the  Internet:  community  access 
to  leaders  and  the  issues.  But  they  considered  very  carefully  the  implications  of  a  trial  of 
an  on-line  forum.  Both  public  and  private  on-line  meetings  would  have  to  be  carefully 
managed  and  would  have  to  conform  to  ethics  standards  including  assuring  all  public 
meetings  were  announced  in  advance  in  various  accepted  venues  and  that  any  private 
meetings  didn’t  result  in  any  thing  that  might  appear  to  be  hidden  collusions.  But  there 
was  one  area  where  a  trial  use  of  the  conferencing  tools  seemed  straightforward  -  the 
local  quality  council  met  privately  to  discuss  issues  of  quality  of  life  in  the  town,  and 
their  current  topic  was  education  standards.  They  were  in  the  process  of  soliciting  expert 
advice  on  education  standards  from  a  town  expert  and  integrating  it  with  their  own  goals 
for  the  community.  Their  vision  was  that  on-line  conferencing  would  allow  ‘he  diverse 
voluntary  members  of  their  council  to  meet  without  traveling  to  a  meeting  place.  Once 
they  had  a  specific  proposal  for  the  community  they  could  present  it  via  an  on  line  forum 
to  the  community  as  a  whole. 

We  discussed  in  detail  each  of  these  groups’  current  usage  of  Internet  Technology  and 
then  educated  the  leaders  about  emerging  conferencing  tools  that  were  on  the  brink  of 
being  ready  to  bring  to  market.  Because  these  community  leaders  had  already  taken 
major  strides  in  the  use  of  the  Internet  within  their  community  they  were  more  ihan  ready 
to  extend  their  visions  of  use  to  the  additional  conferencing  tools. 

Using  the  Tools 

fhese  three  groups  seemed  to  easily  understand  the  idea  of  going  to  a  W\-b  page  wuhin 
the  context  of  an  overall  Web  site,  which  wc  called  the  Forum  Centet.  and  (o  enter  an 
area  where  they  would  meet  with  other  users  in  rooms  similar  to  chat  rooms.  In  fact,  the 
Web  conference  room  made  it  easier  for  them  to  connect  because  it  placed  the 
responsibility  on  each  individual  to  show  up  at  the  right  time,  rather  than  forcing  users  to 
call  each  other  individually  as  in  some  de.sktop  conferencing  systems. 

It  was  important  to  make  meetings  visible  on  the  Web  with  effective  placement  of  links 
and  a  directory  of  meetings,  as  shown  in  Figure  3.  Our  interviews  made  it  clear  that 
meeting  privacy  was  a  critical  need,  despite  the  assumption  of  public  access  that  many 
people  make  about  web  sites.  Users  did  not  seem  bothered  by  the  listing  of  public  and 
private  meetings  in  tfie  .same  place  but  understood  that  there  were  meetings  they  might 
not  be  able  to  attend  unless  they  registered. 
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Status 


Public  Fonuoas 


HbW  tDlitae^ 

March  11. 1996  3:30-5:30  pm:  meeting  led  by  MSi^. 

■  Ongbmg  discussion  mom  and  inifo  ayaiiable  anytinie. 

Do  We  Need  More  River 

Ongoing  discussion  room  and  info  available  anytime . 


Private  Fanizns 


on 


MiH  Touzism 

No’?^:  Meeting  led  by  Clarke  Stevens , 


Vou  7I  A&ied.' 


Figure  3:  The  Communications-  Rich  Web  Site  Forum  Center 


Entering  a  meeting  took  users  to  a  web  page  designed  by  the  meeting  leader.  This  page 
included  a  control  panel  for  access  to  the  desktop  conferencing  tools.  An  example  is 
shown  in  Figure  4. 

The  desktop  conferencing  tools  played  varying  roles  across  the  different  groups 
depending  on  what  they  were  trying  to  accomplish  at  the  time.  All  of  the  groups  used 
audio  conferencing  to  run  the  meetings  and  video  conferencing  to  see  each  others’  faces. 
The  Chamber  of  Commerce  met  with  local  businesses  to  brainstorm  ways  of  improving 
their  site  to  attract  more  tourists  so  they  made  extensive  use  of  the  Web  tour  capability  to 
jointly  review  other  sites.  The  on-line  newspaper  and  radio  station  met  to  evaluate 
methods  for  improving  their  on-line  newspaper  to  attract  more  readership.  They  also 
used  the  Web  tour  but  also  spent  substantial  time  sharing  preexisting  documentation.  The 
Quality  Council  meet  to  compare  expert  education  standards  with  their  own  emerging 
standards  based  on  community  goals.  They  used  the  white  board  extensively  to  diagram 
differences  in  standards. 

Visual  Sharing 

These  three  examples  demonstrate  the  value  of  visual  sharing  at  a  distance.  For  example, 
because  the  Winona  Chamber  of  Commerce  was  in  the  process  of  revising  their  own  Web 
site,  they  shared  photographs  of  visitor  attractions,  movies  etc.  The  City  Council  group 
used  video  clips  to  introduce  absent  members  to  the  team  and  identified  video  tape 
segments  from  other  meetings  to  show  the  team. 
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Figure  4:  A  Forum  "Room" 


Transitions  between  asynchronous  and  real  time  collaboration 

The  groups  also  needed  smooth  transitions  between  asynchronous  and  real  time 
collaboration.  All  groups  used  email  to  communicate  with  each  other.  They  learned  to 
share  much  of  their  planning  documentation  and  messages  during  the  live  meetings.  New 
uses  of  traditionally  asynchronous  tools  emerged  including  meeting  planning  in  bulletin 
boards,  agenda,  photograph,  and  documentation  storage  for  "handouts"  in  the  file  sharing 
tool,  and  use  of  chat  for  side  conversations  during  audio  meetings. 

New  meeting  protocols 

We  observed  new  meeting  protocols  emerge  as  participants  experimented  with  a  variety 
of  new  practices.  They  identified  key  roles  and  responsibilities  for  meeting  management 
and  leadership  as  well  as  for  training  the  group  on  tool  use.  The  collaboration  tools 
themselves  had  few  inherent  turn-taking  mechanisms,  and  participants  explored  a  variety 
of  options  for  coordinating  use  of  the  tools:  for  taking  turns,  speaking,  controlling  the 
white  board,  and  especially  coordinating  multi-party  tours. 


All  three  groups  ended  up  wanting  support  for  both  local  and  extended  group 
collaboration,  ^^ile  they  regularly  met  in  these  local  groups  they  also  all  participated  in 
state-wide  and  national  groups  on  similar  topics  using  overlapping  materials.  Many  of 
the  materials  used  at  the  state  and  national  level  were  already  available  on  the  Web,  and 
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each  group  used  these  materials  as  they  planned  their  own  agendas.  Thus,  extending 
collaboration  to  geographically  broader  groups  seemed  natural. 

These  trial  meetings  with  community  leaders  showed  in  substantial  detail  how  the  real¬ 
time  Internet  collaboration  capabilities  might  be  integrated  into  the  pre-existing 
communication  matrix  of  the  local  community  and  extend  that  community  to  state, 
national,  and  international  forums.  Because  we  worked  with  the  community  as  a  whole 
as  well  as  its  functional  parts,  we  were  able  to  discover  specific  design  requirements  for 
integrating  the  tools  into  the  community’s  existing  web  sites,  methods  for  managing 
privacy  and  open  forum  registration  to  respect  information  boundaries,  and  methods  for 
organizing  sites  so  they  did  not  challenge  various  social  constraints.  We  also  discovered 
needs  for  training  and  consulting  to  help  groups  like  this  not  only  learn  to  use  the 
software,  but  to  understand  how  to  avoid  socially  significant  blunders.  We  anticipated 
that  with  this  depth  of  understanding,  the  community  leaders  would  now  share  more 
detailed  visions  of  the  future  with  the  schools  and  the  families  who  were  quickly  adopting 
the  technology. 

Conclusions 

Traditional  ways  of  identifying  early  adopters  focus  on  individual  characteristics  that 
predispose  one  to  adopting  a  technology.  For  example,  typical  market  segmentation 
studies  reveal  demographic  and  consumer  characteristics  that  describe  likely  adopters 
[10].  In  contrast,  we  found  that  it  is  not  enough  to  identify  individual  early  adopters. 
While  individuals  play  key  roles  in  getting  the  community  to  adopt,  their  efforts  would  be 
futile  without  the  support  and  vision  of  other  entities  in  the  community.  The  adoption  of 
the  Internet  at  the  Senior  Friendship  Center  in  Winona  illustrates  this. 

While  a  key  value  of  the  Internet  for  people  in  Winona  was  in  the  access  it  gave  them  to 
remote  information  resources  and  the  ability  to  communicate  with  distant  experts  and 
family  members,  we  found  a  surprising  amount  of  interest  in  local  communication  and 
information  access  across  all  three  projects.  The  fact  that  there  was  local  information  on¬ 
line  through  the  Winona  Daily  News,  City  Hall,  and  the  schools  was  a  key  driver  in 
adoption  of  the  Internet  for  the  average  Winona  resident.  That  individuals  and 
organization  could  publish  information  to  the  web  and  easily  access  town  officials  was 
empowering.  Familiar  names  and  faces  on  the  web  created  a  friendly  place  from  which 
novices  could  begin  their  journey  into  the  wider  world  of  the  web  and  the  Internet. 

Had  we  not  approached  Winona  as  a  community,  using  an  ethnographic  framework,  we 
never  would  have  understood  the  ways  in  which  its  social  organizations  supported  and 
influenced  each  other  in  adopting  the  Internet.  It  would  have  been  difficult  to  identify  key 
players  and  organizations  that  would  be  open  to  our  scrutiny  throughout  the  duration  of 
the  Virtual  School  and  Communications  Rich  Web  Site  projects.  Our  long-term  presence 
in  the  town  gave  us  an  understanding  of  the  complex  interrelationships  between 
individuals  and  social  organizations,  and  it  helped  us  understand  nuances  in  the  content 
of  their  communications. 

Working  with  communities  before  technology  introduction  can  provide 
telecommunications  companies  key  insights  about  needs  for  new  telecommunication 
technologies  at  the  individual,  group,  and  community  level.  Working  with  a  community 
as  Internet  technologies  were  introduced  gave  us  opportunities  to  observe  reactions  to  this 
new  communications  medium.  The  implications  for  business  are  not  only  understanding 
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preliminary  requirements  for  services  at  the  three  levels,  but  also  understanding  strategies 
for  introducing  and  marketing  the  technology  to  communities  during  and  after  their  first 
adoption.  Because  these  strategies  derive  from  community-based  research,  they  can  take 
into  account  the  social  structure,  vested  interest,  and  temporary  deterrents  of  the 
community,  and  can  support  the  integration  of  technology  into  the  community  through 
vehicles  such  as  help  desks,  training  packages,  user  group  support,  and  consulting 
services. 
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Digital  Multimedia  Broadcasting  Transmission  of  Video 
and  High  Data  Rate  Signals  into  Moving  Vehicles 


Prof.  Dr.  Gert  Siegle,  Robert  Bosch  GmbH,  Hildesheim 


Abstract 

Broadcasting  Services  (Radio  (Audio  only)  and  TV)  are  the  last  telecommunication 
services  to  be  changed  from  analog  to  digits  transmission  schemes.  Within  an  EUREKA- 
project  (EU  147)  a  Digital  Audio  Broadcasting  standard  (DAB)  has  been  defined  and 
agreed  upon.  Field  trials  have  started  as  soon  as  1995,  already.  Decisions  for  introduction 
have  been  taken  in  UK,  Canada  and  the  United  States,  already.  In  Germany  regular 
services  start  end  of  1997,  So,  by  the  end  of  this  year  about  100  Mio  people  worldwide 
will  have  access  to  DAB,  already. 

Its  merits  are  Performance  and  quality  comparable  to  CD-reproduction,  even  for  mobile 
reception  at  up  to  300  km/h,  no  deterioration  from  multipath  reception  and  interference 
effects,  and  reserves  for  many  data  services  available. 

DAB  uses  a  regular  7  or  8  MHz  -TV-channel  which  is  divided  in  4  blocks  of  1,5  MHz 
each,  allowing  a  gross  data  rate  of  2,3  Mbit/s,  6  audio  programs  can  be  put  into  every 
block.  Instead  of  6  audio  programs  just  one  digital  TV-program  of  about  1,3  Mbit/s  for 
the  video  can  be  transmitted.  This  allows  for  the  very  first  time  to  receive  a  clear  and 
absolutely  stable,  undisturbed  TV-picture  or  other  digital  data  stream  of  up  to  about  1,5 
Mbit/s  net  even  in  moving  vehicles  -  the  solution  to  a  problem  which  is  more  than  60 
years  old.  So  a  new  and  growing  group  of  customers  can  be  reached  which  have  not  been 
accessible,  yet  -  the  mobile  customers. 

Starting  some  time  later,  Digital  Video  Broadcasting  for  terrestrial  distribution  (DVB-T) 
has  been  designed  using  about  the  same  schemes  as  DAB  but  with  a  higher  data  rate  and 
greater  bandwidth  requirement.  It  is  not  capable  for  mobile  reception  at  higher  speeds, 
yet,  DAB  has  been  installed,  already.  In  case  further  transmitters  chains  for  DVT-T  are 
intended  to  be  installed,  we  recommend  a  convergence  of  DAB  and  DVB-T  to  a 
compatible  standard  is  recommended. 

In  any  circumstance,  only  "Digital  Multimedia  Broadcasting"  explains  all  the 
possibilities,  a  new  digital  standard  can  cover:  Text,  voice,  music,  still  frame  pictures, 
television  and  all  other  one-way-data  services.  The  standards  will  be  explained  and 
compared.  For  DAB/DMB  hardware  products  and  software-services  will  be  demonstrated 
for  DAB/DMB  in  operation. 
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Introduction 

Surveys  have  revealed  that  highly  mobile  people  nowadays  are  most  interested  in  having 
the  services  they  use  at  home  or  at  the  office  available  to  diem  while  on  the  road,  too.  The 
widespread  use  of  paging  and  mobile  phones  proves  this  -  or  the  well  established 
reception  of  AM  and  FM  radio  broadcasting  in  motorcars. 

In  towns  or  hilly  areas,  the  terrestrial  transmission  of  voice  or  music  into  moving  vehicles 
suffers  sometimes  from  interference  effects  due  to  multipath  propagation:  The  signal  is 
fading,  and  the  amplitude  of  the  reproduced  sound  may  oscillate  or  sometimes  even 
vanish. 

This  is  disturbing  despite  the  fact  that  most  often  the  human  ear  helps  to  smoothen  -  it 
integrates  and  levels  off  effects  of  less  than  about  20  ms  duration. 

But  even  if  the  voice  is  rather  well  received,  the  transmission  of  signals  which  are 
sensitive  to  short  drop-outs  and  interference  effects  often  give  poor  results:  A  fax 
transmitted  via  the  same  audio  channel  may  be  blurred  from  horizontal  dark  lines  if  drop¬ 
outs  just  cancelled  those  bits  which  control  the  line  synchronization. 

The  reception  of  TV- signals  is  even  worse  if  there  is  some  multipath  propagation:  Line 
and  frame  synchronization  get  instable,  the  color  disappears  from  time  to  time,  the  picture 
becomes  unclear  from  ghosts. 

Obviously:  The  international  standards  for  analog  TV-distribution  (NTSC,  PAL, 
SECAM)  are  totally  inadequate  to  transmit  television  into  fast  moving  vehicles  or  even  to 
portable  television  sets  while  they  are  carried  around. 

Consequently,  up  to  now  the  mobile  user  cannot  be  served  with  TV  or  any  high  data  rate 
digital  service.  So,  a  totally  new  transmission  scheme  has  to  be  applied.  We  found  a 
multi-tiered  solution  which  masters  the  problems: 

•  digitization  and  strong  data  compression  of  audio  and  video  signals, 

•  transmissions  using  a  modulation  which  is  insensitive  to  multipath  propagation 
(COFDM  -•  Coded  Orthogonal  Frequency  Division  Multiplexing), 

•  use  of  unoccupied  frequencies  (taboo-channels)  within  the  broadcast  bands  which  is 
possible  because  then  the  COFDM-modulation  does  not  disturb  the  established 
television  programs,  and 

•  receivers  for  the  users  which  allow  reception  and  decoding  of  these  new  types  of 
signals. 

This  is  basically  an  extension  of  the  Digital  Audio  Broadcasting  (DAB)  -  a  new  radio 
system  specifically  developed  to  serve  the  mobile  user:  A  television  channel  is  divided  in 
blocks  of  1.5  MHz  bandwidth  each.  Every  block  can  then  transport  a  gross  data  rate  of  up 
to  2,3  Mbit/s,  Error  protection  reduces  this  to  about  1.7  Mbit/s.  For  audio  transmission 
this  data  rate  is  then  split  into  6  audio  channels  each  of  392  kbit/s.  The  audio  compression 
according  to  ISO  MPEG  11172  is  so  well  performing  as  to  allow  then  a  sound 
reproduction  comparable  to  the  quality  of  CD.  The  modulation  scheme  makes  the 
wireless  transmission  as  good  and  secure  as  through  a  wired  line. 

Digital  Audio  Broadcasting  widens  to  ” Digital  Multimedia  Broadcasting” 

We  made  this  DAB-system  more  versatile  using  the  full  data  stream  as  a  "reliable 
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wireless  data  highway”  of  up  to  1.7  Mbit/s.  This  is  30  times  faster  than  wire-bound  ISDN, 
60  times  faster  than  fax,  and  about  40  times  faster  than  the  serial  input  of  most  PCs 
accepts. 

With  1.7  Mbit/s  a  complete  television  signal  can  be  transmitted  when  compressed  firstly 
using  the  internationally  standardized  MPEG  1  and  MPEG  2  schemes  (Motion  Picture 
Expert  Group).  We  use  0.94  Mbit/s  for  the  picture  and  again  392  kbit/s  for  the  sound. 
This  results  in  1. 152  Mbit/s  for  the  total  package.  Error  protection  bits  fill  the  data  stream 
up  to  the  1.7  Mbit/s. 

The  reception  with  mobile  receivers  is  convincing:  Stable  picture,  no  flicker,  no 
synchronization  defects,  no  switching  from  color  to  monochrome. 

The  subjective  picture  quality  is  often  better  even  than  with  PAL  or  NTSC:  Outlines  and 
color  transitions  do  not  show  cross-color  distortion  -  as  otherwise  seen  on  newsreader’s 
striped  jacket.  Colored  noise  as  noticed  generally  in  large  areas  of  a  single  color  is  also 
gone.  Even  if  the  mere  dot  resolution  is  comparable  to  the  output  of  video  recorders,  the 
excellent  color  contrast  and  the  sharp  outlines  make  a  "brilliant”  picture. 

It  is  obvious  that  the  system  can  handle  all  possible  digital  data  streams  -  not  only  for 
sound  but  also  for  all  other  multimedia  appHcations  such  as  text,  still  frame  or  slowly 
moving  pictures,  video  and  television. 

So  the  name  "Digital  Multimedia  Broadcasting"  (DMB)  is  describing  very  well  these 
overall  capabilities. 

DAB  infrastructure  can  be  used  for  DMB 

The  DAB -transmitters  can  be  used  for  the  transmission  of  moving  pictures.  Additional 
equipment  are  a  MPEG  2  source  encoder,  inserter  for  the  channel  error  protection  and  a 
multiplexer  to  combine  the  sound  and  vision  data  streams. 

Mobile  DMB  receivers 

The  receivers  developed  by  Bosch  and  delivered  under  the  Blaupunkt  brand  name  to 
numerous  projects  in  interested  countries  were  designed  already  to  decode  the  total  data 
stream  out  of  the  1.5  MHz-block. 

The  complete  system  consists  at  the  time  being  from 

•  a  modified  DAB -receiver  with  integrated  decoders  for  sound  and  vision  to  be 
mounted  into  the  trunk  compartment  because  the  size  of  these  very  first  sets  are 
comparable  to  a  CD-changer, 

•  a  regular  Blaupunkt  car  radio  for  tuning,  volume  adjustment  and  a  bus  interface  to 
control  the  DAB/DMB -receiver, 

•  one  ore  more  color  monitors  to  be  mounted  within  the  vehicle,  and 

•  a  single  rod  antenna  (no  antenna  diversity  required  any  more!). 

Still  frame  or  even  moving-picture  information  for  drivers? 

Critics  obviously  will  ask:  Do  people  need  full-motion  picture  reception  in  their  cars? 
Wouldn’t  it  distract  the  driver?  Well,  no  one  thinks  that  any  driver  should  watch 
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television  while  driving.  But  being  able  to  watch  news,  information  or  entertainment 
programs  while  riding  on  public  transportation  such  as  busses,  subways  or  trams,  or  in 
trains  or  even  in  the  back  seat  of  a  van  would  be  an  interesting  alternative  for  passengers. 

This  new  DMB-system  also  opens  up  opportunities  for  providing  drivers  with  relevant 
information  in  visual  or  spoken  form:  Traffic  conditions,  recommended  detours, 
departure  schedules  for  public  transport  utilities,  loading  of  parking  facilities  and  a  wealth 
of  other  useful  information. 

So,  autonomous  navigation  and  route  guidance  systems  in  cars  will  become  even  more 
effective  if  their  digital  data  base  gets  updated  automatically  or  when  entering  a  town  - 
the  fastest  route  will  then  be  selected  avoiding  construction  sites  or  traffic  jams. 

In  any  case,  an  adequate  display  of  a  navigation  system  may  be  very  well  used  as  the 
video  output  of  the  Digital  Multimedia  Broadcasting,  too. 

Introduction  of  Digital  Multimedia  Broadcasting 

In  many  countries,  DAB  is  already  decided  to  get  introduced  as  an  audio  service.  So  the 
extension  to  Digital  Multimedia  Broadcasting  (DMB)  as  a  digital  broad  band  service  is 
rather  straightforward.  Digital  Video  Broadcasting  (DVB)  as  being  propagated  via 
satellite  or  via  cable  does  not  allow  mobile  reception. 

The  acceptance  of  the  mobile  DMB-services  by  the  customers  will  grow  with  the 
availability  of  programs  and  with  the  coverage  of  the  country  with  regular  services.  The 
introduction  will  go  even  faster  if  specific  applications  and  clients  are  served  firstly. 

So  we  install  networks  and  regular  services  for  the  information  and  the  entertainment  of 
people  travelling  in  railways,  subways,  taxis,  busses  or  on  boats  and  ferries.  Most  actual 
traffic  information  such  as  departure  times  of  trains  or  airplanes  will  be  transmitted  as 
well  as  news,  stock-exchange  data,  weather  forecasts,  advertising. 

From  this  the  potential  users  get  well  acquainted  with  the  new  possibilities.  The 
consumer  market  will  grow  even  faster  as  soon  as  truly  portable,  battery  operated  sets 
will  become  available. 

We  expect  that  DMB  may  then  gradually  replace  other  terrestrial  television  transmission 
because  it  makes  no  sense  to  operate  one  set  of  transmitters  just  to  serve  the  fixed 
receivers  when  another  transmission  scheme  can  serve  both  the  mobile  and  the  stationary 
receivers. 

In  this  situation  our  idea  may  be  helpful  to  transmit  via  the  same  television  transmitter 
both  a  regular  NTSC-program  and  -  filled  into  the  gap  between  sound  carrier  and  next 
higher  picture  carrier  -  a  digital  one  of  limited  bandwidth.  This  has  still  to  get  formal 
approval  but  can  ease  the  DMB -introduction  significantly. 

Demonstration 

Figures  1  shows  the  quality  of  a  PAL-Signal  as  received  in  a  moving  vehicle.  Figure  2 
shows  the  same  picture  if  transmitted  and  received  using  DMB.  During  the  conference  it 
is  planned  to  have  an  on-site  demonstration  of  the  DMB-system  in  a  driving  vehicle. 
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Figure  1:  Reception  of  PAL-Signals  in  Moving  Vehicles 


Figure  2:  Digital  Multimedia  Broadcasting  -  Reception  in  Moving  Vehicles 


The  Interface  -  An  Invitation  To  Communication 


Input  &  Output  Devices  for  the  Human  Body  in  VR  Environments 

Monika  Fleischmann 

GMD  -  German  National  Research  Center  for  Information  Technology 
Schlofi  Birlinghoven,  D-53754  Sankt  Augustin 

Phone  +49-2241-14-2366/2809,  Fax  +49-2241-14-2040 
e-mail  fleischmann@gmd.de 


Introduction 

The  speed  at  which  information  is  conveyed  and  the  quantities  involved  can  be 
overwhelming  for  the  individual.  Can  the  arts  work  with  science  and  technology  to 
counteract  this  feeling  of  disorientation?  To  combat  disorientation  and  the 
omnipresence  of  technological  power  we  resolve  to  design  artificial  worlds  in  which 
we  can  ’navigate’  as  if  in  a  game.  Interactive  artistic  work  is  a  synthesis  of  theory, 
technology,  design  and  play.  In  this  paper  different  human  machine  interfaces  are 
described  and  various  fields  of  application  are  presented,  showing  scenarios  already 
realized  as  well  as  a  variety  of  possibilities  for  future  use. 

Principles 

Our  work  is  based  on  recounting  a  story  in  space.  Three  elements  determine  the 
structure  -  the  spectator,  the  story  and  the  (virtual)  space.  The  area  of  conflict  is  the 
relationship  that  exists  between  man  and  his  world  of  experience.  The  immersion 
into  a  story  which  is  presented  virtually  and  interactively  as  a  framework  for  human 
actions  has  a  fundamentally  different  purpose  than  film  or  theatre.  The  basic  form  of 
the  theatre  is  dialogue  and  contemplation  of  the  various  ways  of  thinking  and 
behaving.  Film  is  the  modem  form  of  dramatic  art  and  represents  a  progression  from 
the  static  image  to  the  moving  one.  Interactive  media  art  has  the  task  of  bringing  the 
qualities  of  the  theatre  and  film  into  confrontation  and  of  intermeshing  these 
qualities.  The  audience  is  given  an  unusual  responsibility  in  this  regard.  They  play  an 
important  role  in  the  story.  The  poetry  of  the  interface  determines  the  dimension  of 
interaction  and  one’s  own  experience.  What  am  I  experiencing,  feeling?  Stroking  my 
hand  over  the  digital  water,  coming  face  to  face  with  myself,  flying  through 
surrealistic  landscapes,  looking  from  a  bird’s  or  a  mouse’  point  of  view,  crossing  real 
and  virtual  space.  What  does  it  all  mean?  The  visitor  is  invited  to  examine  these 
questions.  It  is  an  invitation  to  communication. 

Basic  concepts 

Virtual  Environments  still  suffer  from  input  and  output  devices  that  work  against  the 
human  body  and  mind.  The  body  with  its  communication  channels  (skin,  eyes,  ears, 
voice)  opens  imagination  spaces.  Body  motion  is  fundamental  to  learning  and  living. 
Interaction  and  feedback  intensify  capabilities  of  perception.  Artistic  dance,  medical 
and  therapeutic  methods  [FEL78]  aim  to  improve  the  awareness  and  the  movability 
of  the  body. 

The  authors’  main  interest  is  to  develop  systems  that  focus  on  the  immersion  of  the 
viewer  in  a  virtual  environment  without  restrictions  like  heavy  interface  devices  and 
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cables.  This  aim  is  also  reflected  in  their  earlier  work:  In  Berlin  -  Cyber  City 
[STRA91]  we  play  the  "let  you  finger  do  the  walking"  game  and  use  an  electronic 
thimble  (Polhemus)  to  move  around,  show  and  visualise.  With  the  Spatial  Navigator 
[FLE95]  we  play  "looking  with  the  feet"  and  use  a  treadmill  where  the  viewer  walks 
through  a  virtual  castle.  The  Responsive  Workbench  [KRU95]  is  "thinking  with  the 
hand".  The  Workbench  has  put  the  conventional  dialogue  concept  for  man-machine 
communication  into  a  user-oriented  shape.  Virtual  objects  and  tools  are  projected 
stereoscopically  on  a  real  workbench  as  a  virtual  design  environment.  The  Skywriter 
[STRA96]  is  able  to  fly  through  virtual  landscapes  without  any  effort.  To  do  this,  he 
uses  neither  mouse,  joystick  or  virtual  reality  glove,  but  a  Virtual  Balance  [FIJE97]. 
This  is  a  platform  with  3  weight  sensors,  which  is  controlled  solely  by  the  changes  in 
the  position  of  the  human  body’s  centre  of  gravity.  The  computer  art  installations 
Rigid  Waves  [FLE96],  a  mirror  based  on  image  processing  and  Liquid  Views 
[FLE93],  a  water  surface  based  on  a  touch  screen  stride  new  ways  in  man-machine- 
communication,  where  the  interface  with  the  machine  is  imperceptible. 

Conclusion  and  Outlook 

The  key  research  and  development  issues  are  not  just  the  design  of  better  individual 
interfaces  to  information,  but  rather  the  design  of  spaces  of  activity  that  enable 
people  to  interact  with  information  and  each  other  in  new  ways.  The  spaces  need  not 
depend  on  geographical  proximity  of  the  participants  and  thus  have  the  potential  for 
enabling  new  forms  of  meaningful  collective  interaction  to  take  place  on  a  global 
scale. 

For  further  development  we  will  observe  the  following  directions: 

•  the  unconscious  navigation  with  free  movement  of  the  body 

•  the  conscious  navigation  with  directed  movements 

•  the  navigation  of  two  or  more  people  in  a  distributed  virtual  environment. 

Future  applications  may  include: 

Marketplace/Teleshopping 

•  to  navigate  the  World  Wide  Webspace  (VRML) 

•  to  surf  through  desired  shopping  malls 

•  Games/Scepe  parks 

•  to  offe^teractive  entertainment  in  distributed  virtual  worlds. 

•  to  traimcSdperative  behaviour  for  navigation 

•  to  invenf  a’  panoramic  environment  for  real  or  virtual  worlds 

•  Museums/Leaming  Centers 

•  to  pre^^t  the  simulation  of  real  and  artistic  worlds 

•  to  (Jevelop  interactive  3D  learning  environments. 

•  Hospitals/Medical  Training  Centers 

•  f(/^tfain  .motoric  disabled  people 

supporf-a  dialogue  with  autistic  people 

•  fd  improve  patients  suffering  on  disturbance  of  equilibrium 
,•  Performance 

•  to  train  the  body  for  non-oral  communication 

; " ' •  to  find  new  body  expressipn  for  choreography 
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The  development  of  future  communication 
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Abstract 

o.tel.o,  the  telecommunication  company  of  RWE  and  Veba  is  carrying  out  a  multimedia 
trial  in  the  western  part  of  Germany.  The  cities  Cologne,  Diisseldorf,  Gelsenkirchen  and 
Dortmund  are  included  in  this  technical  and  economical  trial  of  a  multimedia  network 
and  its  services. 

This  trial,  which  is  organized  by  the  ,>tultimedia  Competence  Center^  at  o.tel.o,  consists 
of  two  basic  parts,  the  realization  of  hard-  and  software  on  one  hand  and  the  contents  and 
acquisition  of  research  on  users"  behaviour  on  the  other  hand. 

The  realization  of  the  technical  part  is  devided  into  the  design  of  a  wide  area  network 
(WAN)  and  a  metropolitan  network  (MAN)  with  a  high  bandwidth  based  on  TCP/IP  over 
ATM. 

The  realization  of  the  non-technical  part  consists  of  the  acquisition  of  multimedia 
content  including  features  such  as  realaudio,  videostreaming  applications,  video  on 
demand,  teleconferencing  and  telecooperation  and  the  research  of  users"  behaviour  when 
interacting  with  this  content.  The  determinition  of  the  main  application  for  customer 
care,  billing  and  accounting  provides  for  the  right  choice  of  a  biling  and  accounting 
software  tool. 

This  multimedia  trial  will  be  the  based  on  an  open  market-place  in  order  to  locate 
different  contents  and  offers  into  the  internet  network,  which  uses  the  capacity  of  the 
whole  network. 

Introduction 

The  future  will  provide  us  with  many  changes  and  progress  in  telecommunications  and 
its  lines  of  business  like  multimedia  and  online  services.  The  technical  development  in 
this  area  of  telecommunications  changes  so  quickly  that  the  market  of  multimedia  is  still 
not  ready  for  a  use  of  these  professional  services.  Therefore,  o.tel.o  communications 
GmbH,  the  telecommunication  company  of  Veba  and  RWE,  is  carrying  out  a  multimedia 
trial  in  Gelsenkirchen,  which  is  called  „Pilotprojekt  Multimedia  Gelsenkirchen“,  a  city  in 
the  Ruhr  area  in  the  western  part  of  Germany  and  a  trial  in  Cologne,  Diisseldorf, 
Gelsenkirchen  and  Dortmund,  which  is  called  ,4nfoCity  NRW“. 

Both  trials  which  are  organized  in  the  ,Mulimedia  Competence  Center“  of  o.tel.o  consist 
of  the  two  basic  parts,  the  realization  of  hard-  and  software  on  the  one  hand  and  the 
contents  and  acquisition  of  research  on  users"  behaviour  on  the  other  hand. 
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The  magic  triangle 

The  magic  triangle  of  future  multimedia  communication  consists  of  three  principal 
ingredients:  the  content,  the  bandwidth  and  the  local  loop.  Those  three  edges  are 
connected  by  dependencies. 


Figure  1.  The  relationship  of  content,  bandwidth  and  local  loop 

The  content  and  the  bandwidth  of  the  transport  network  as  well  as  the  multimedial 
content  are  bundled  together  to  an  inseparable  unit. 

Every  content  should  be  a  value  added  content.  The  creation  of  multimedial  contents  like 
videos  is  quite  expensive  so  the  claim  at  a  value  added  multimedial  content  is  very  high. 
Not  every  content  has  enough  value  to  be  presented  as  a  video  clip  but  the  lines  of 
business  like  holidays,  teleshopping  or  infotainment  are  attractive  multimedial  contents. 

In  order  to  transport  multimedial  contents  to  the  customer,  the  network  performance 
should  have  a  broad  bandwidth  vrithout  any  bottleneck.  As  well  the  infrastructure  of  this 
multimedial  network  should  use  existing  lines  like  coax-networks  or  twisted  pair 
networks  in  order  to  lower  the  investment  costs  of  the  whole  network.  Mostly  the 
network  leads  to  a  hybrid  network  (HFC=hybrid  fibre  coax). 

The  relationship  between  the  local  loop,  i.e.  the  customer  in  a  regional  area,  and  the 
content  leads  to  regional  services.  A  regional  content  provider  enters  his  regional 
information  and  services  into  the  network  so  that  the  customer  is  supported  in  his 
everyday  hfe. 
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The  main  task  of  the  near  future  is  the  construction  of  a  bandwidth  network  to  the 
customer  and  the  transfer  of  important  content,  which  must  be  interesting  enough  for  the 
customer  to  spend  some  money  for  this  value  added  information. 

Realization  of  a  future  backbone-network 

The  trial  has  the  goal  to  transfer  data  over  a  local  and  metropolitan  network  on  a  fibre 
network  with  155  Mbit/s  up  to  622  Mbit/s. 

Depending  on  the  distance,  different  methods  of  transportation  are  available: 

a)  wide  area  network  (WAN) 

•  use  of  pipline  routes 

•  use  of  railway  routes 

•  use  of  high  voltage  overhead  lines 

b)  metropolitan  area  network  (MAN) 

•  city  carrier 

•  other  telephone  companies  through  interconnection 

•  other  companies  with  private  networks  through  interconnection 

The  integration  of  the  potential  network  supply  depends  on  the  position  of  the  partners.  It 
is  easier  to  cooperate  with  partners  who  have  large  networks  and  who  are  interested  in 
using  their  routes  additional  by  another  medium.  Therefore  big  power  supply  companies 
are  quite  interested  in  this  kind  of  cooperation. 

The  high  voltage  overhead  network  of  RWE  with  dark  fibre  covers  the  heavily  inhabited 
areas  of  the  rivers  Rhein  and  Ruhr,  o.telo  communications  can  utilize  the  lines  of  RWE 
Energie  and  PreuBen  Elektra,  the  power  supplier  of  Veba.  The  network  of  the  trial  of 
o.tel.o  consists  of  the  dark  fibre  lines  of  different  city  carriers  RWE  Energie  AG  and  the 
German  railway  company,  Deutsche  Bundesbahn,  which  has  a  private  fibre  network  for 
own  use. 

System  configuration  of  a  metropolitan  area 

The  city  network  of  Gelsenkirchen  is  operated  by  the  municipal  city  supplier,  the 
Stadtwerke  Gelsenkirchen  and  RWE  Energie.  It  appears  that  every  new  private  carrier 
who  has  no  lines  of  its  own  is  forced  to  cooperate  with  city  carriers  to  supply  their  new 
clients  with  value  added  services. 
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Gelsenklrch«n 


Figure  2  InfoCity  NRW  backbone  based  on  the  routes  of  the  german  railway  company 


Figure  3  system  configuration  of  Gelsenkirchen 
Future  services  over  a  network  expect  a  broad  bandwidth  connection  to  the  customer.  All 
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services  like  data,  voice  or  video  are  mixed  on  one  network  because  there  will  be  no 
difference  in  the  transport  medium  of  a  fibre  line. 
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Chart  L  Future  services  need  a  bandwidth  network 

All  digital  data  is  based  on  a  package  oriented  service  (e.g.  TCP/IP  over  ATM).  ATM 
(asynchronous  transfer  mode)  is  suitable  for  this  mixture  of  different  services.  ATM  is  a 
transmission  protocol  (second  level  of  the  ISO/OSI-model)  which  is  used  in  WAN  and 
MAN  networks,  but  still  not  in  connection  of  residential  clients.  In  Germany  a  lot  of 
residential  districts  are  connected  to  a  coax-network  which  supplies  the  households  with 
television  and  radio.  This  trial  uses  this  network.  This  requires  that  the  amplifiers  must 
be  changed  into  amplifiers  with  a  backchannel.  The  base  of  this  network  for  these 
heterogeneous  services  is  a  hybrid  (coax/fibre)  network. 

The  „last  to  the  customer 

The  solution  of  the  last-mile-problem  deals  with  the  harmonization  between  the 
customer's  access  with  a  broad  bandviddth  and  a  lot  of  individual  convenient  value  added 
services.  The  following  chart  shows,  which  bandwidth  is  necessary  to  supply  the 
different  value  added  services. 

Convenient  content  as  a  base  for  permanent  acceptance 

The  future  service  providers  expect  a  broad  bandwidth  connection  to  their  clients.  Apart 
from  the  technical  realization  only,  the  ,4ght“  content  will  lead  to  permanent  customer's 
acceptance.  But  a  killer  application  of  the  ,4ght“  content  does  not  exist.  The  physical 
network  with  its  equipment  forms  a  plattform  for  an  „open  market-place“  for  aU 
providers  of  services  and  information.  That  means  the  broad  bandwidth  network  is  still 
the  basic  instrument  to  spread  contents  like  information,  advertising  or  other  value  added 
services.  The  consumer  needs  a  complete  portfolio  of  online  services  including  solid 
accounting,  billing  and  customer  care  for  a  permanent  relationship. 

The  system  will  succeed  economically  if  the  provider  is  able  to  solve  the  technical 
problem  of  broad  bandwidth  as  well  as  the  problem  regarding  the  sensible  use  of 
multimedia  and  interactivity. 
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1  necessary  bandwidth 

value  added  services 

downstream 

uDstream 

house  management 
(light,  house  alarm,  heating) 

<  64  kbit/s 

<  64  kbit/s 

management  system  of 
supply  activities 
(remote  meter  reading, 
energy  mangagement 

<  64  kbit/s 

<  64  kbit/s 

picture-phone 

(value  added  phone  services) 

>  64  kbit/s 

>  64  kbit/s 

(interactive)  television 
(video  on  demand,  broad¬ 
casting,  entertainment,  value  added 
teleshopping,  telebanking 

>  1,5  Mbit/s 

<  64  kbit/s 

high  fidelity  set 

(real  audio,  audio  on  demand) 

<  622  kbit/s 

<  64  kbit/s 

personal  computer 
(teleshopping,  telebanking,  internet 
services,  videoconferencing) 

>  1,5  Mbit/s 

>  1,5  Mbit/s 

services  for  older  or  sick  people 
(videoconferencing,  alarm  systems) 

>  1,5  Mbit/s 

>  1,5  Mbit/s 

Chart  2.  convenient  value  added  services 


The  main  ideas  for  a  continous  customers acceptance  can  be  summerized: 

•  the  content  leads  to  value  added  service  for  every  user,  who  pays  for  the  services 

•  customers 's  data  is  safe,  personal  data  will  be  encrypted  and  authentified 

•  the  bandwidth  will  be  dynamically  allocated  by  a  management  system 

•  the  infrastructure  of  fibre  lines  and  TV-cables  is  able  to  be  payed  by  the  local  players 
who  rent  the  lines  for  their  customers 

The  main  problem  of  future  businesss  is  the  generation  of  value  added  contents  and  a  big 
number  of  customers  paying  for  the  service.  The  more  customers  are  interested  in  the 
service  the  more  providers  will  offer  their  content  on  the  network. 

Conclusions 

The  results  of  this  big  trial  of  o.tel.o  will  provide  the  answers  to  all  these  questions  so 
that  we  can  still  expect  that  the  future  will  bring  us  a  variety  of  telecommunication  links 
which  use  broad  bandwidth.  Today  we  can  establish  that  the  market  for  all  these  services 
is  correlated  to  the  offer  of  value  added  contents  and  the  drop  of  the  price  for  its  use. 
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High  quality  content  needs  an  infrastructure  with  a  high  bandwidth  and  a  users^ 
acceptance,  to  spend  money  for  these  value  added  services.  Tlie  future  will  bring  us  a 
change  of  content  which  needs  a  narrow  bandwidth  to  a  content  with  broad  bandwidth 
full  of  multimedial  components.  Both  narrow  and  broad  bandwidth  content  will  coexist 
in  dependence  of  its  special  application. 
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Abstract 

Most  of  today’s  commercially  available  virtual  environment  systems  place  great  empha¬ 
sis  on  vision  but  still  fail  to  adequately  address  the  auditory  and  tactile  domains.  Since 
virtual  environments  are,  by  their  very  nature,  multi-modal  applications,  appropriate  con¬ 
sideration  of  modalities  other  than  the  visual  modalities  is  vital  in  order  to  achieve  a  con¬ 
vincing  perception  of  immersion.  The  effect  of  immersion  can  significantly  be  improved 
by  including  auditory,  tactile/thermal  and  force-feedback  perceptions.  The  virtual  envi¬ 
ronment  system  developed  within  the  ESPRIT  basic  research  project  #6358  SCATIS 
{Spatially  Coordinated  Auditory /Tactile  Interactive  Scenario),  funded  by  the  European 
Union,  has  demonstrated  the  feasibility  of  an  immersive  auditoiy/tactile  virtual  environ¬ 
ment  implemented  as  a  modular  multi-process  simulation  system  running  on  a  heteroge¬ 
neous  UNIX  network.  It  was  particularly  designed  as  a  tool  for  multi-modal  psycho¬ 
physics  research  and,  furthermore,  it  was  built  to  serve  as  a  platform  for  future  refine¬ 
ments  and  optimization  of  the  simulation-  and  signal  processing  methods  applied.  Some 
aspects  of  the  implementational  concept  with  a  particular  emphasis  on  the  auditory  sub¬ 
system  as  well  as  the  major  results  achieved  within  the  project  in  terms  of  physical  per¬ 
formance  and  perceptual  parameter  thresholds  are  presented  in  this  paper.  Experiments 
carried  out  with  the  so-called  SCAT-Lab  yielded  quite  promising  results  with  respect  to 
the  auditory  subsystem  whereas,  due  to  technological  difficulties,  the  tactile  effector 
hardware  turned  out  to  be  yet  widely  insufficient  to  render  plausible  perceptions  of  vir¬ 
tual  objects. 

Introduction 

VR  systems  provide  the  potential  to  expose  human  beings  to  a  computer-generated 
and/or  computer-controlled  artificial  environment  in  a  way  that  they  feel  and  act  as  if 
they  were  in  their  natural  environment.  Many  otherwise  complicated  tasks  can  be  per¬ 
formed  intuitively  in  such  a  state  of  mind.  We  call  the  impression  of  a  subject  of  feeling 
present  in  a  natural  environment  while  actually  being  exposed  to  a  virtual  one: 
Telepresence.  In  order  to  achieve  Telepresence,  physiologically  adequate  signals  have  to 
be  delivered  to  the  subject’s  senses.  The  most  important  senses  to  be  addressed  in  this 
context  are  vision,  audition  and  the  tactile  modality  -  plus  kinesthetic  coordination. 

Today’s  commercial  VR  systems  are  seen  to  place  a  great  emphasis  on  vision,  but  they 
do  not  yet  adequately  deal  with  the  auditory  and  tactile  domains  -  although,  to  be  sure, 
these  domains  essentially  enhance  Telepresence  or  can  even  create  it  by  themselves.  It  is 
not  a  case  of  these  facts  not  having  been  understood  or  of  sophisticated  auditory  and/or 
tactile  components  being  deliberately  omitted.  It  is  rather  that  there  is  a  significant  deficit 
in  specialized  basic  knowledge  &  experience  concerning  these  two  modalities  and  in  the 
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technology  necessary  to  deal  with  them. 

Therefore,  general  aim  of  SC  ATIS  was  to  perform  specific  basic  research  in  order  to  in¬ 
vestigate  the  role  of  the  auditory/tactile  modalities  with  respect  to  Telepresence  in  a  se¬ 
lective  and  detailed  way.  To  this  end,  an  interactive  experimental  scenario  was  specified 
as  follows: 

"A  subject  is  exposed  to  a  virtual  space  with  various  (invisible)  auditory/tactile  objects 
distributed  in  it.  He/she  will  localize  and  identify  these  virtual  objects  auditorily,  and  be 
able  to  reach  towards  them  and  grasp  them  individually.  Upon  tactile  contact,  contour, 
texture  and  thermal  attributes  of  the  virtual  objects  will  be  perceived.  It  is  the  task  of  the 
subject  to  manually  move  the  objects  around,  i.e.  to  re-arrange  their  spatial  position 
and  orientation  according  to  an  experimental  plan.  Auditory  feedback  is  given. " 

This  scenario  was  considered  to  be  representative  of  a  variety  of  engineering  problems  in 
Multi-Media  and/or  VR  applications,  for  example:  control,  guidance,  surveillance,  access 
or  evaluation  tasks  -  not  to  mention  training,  education  and  entertainment. 

Vision  was  not  explicitly  addressed,  but  rather  omitted  most  of  the  time  in  this  project 
for  the  following  two  reasons:  Visual  cues  to  the  experimental  subject  may  cause  distrac¬ 
tion  and  mask  effects  due  to  sensory  dominance.  This  would  thus  affect  the  selective 
analysis  of  the  auditory/tactile  modalities.  Furthermore,  there  are  many  application  pos¬ 
sibilities  in  Information  Technology  for  straight  auditory/tactile  virtual  environment 
technologies,  e.g.,  in  situations  where  the  eyes  are  busy  or  where  vision  is  impeded,  im¬ 
paired  or  not  available  at  all.  This  applies  especially  to  Augmented  Reahties  where  e.g. 
warning  signals  or  control  parameters  in  an  airplane  can  be  presented  auditorily  so  as  to 
ease  the  pilot’s  visual  sensory  channel.  Nevertheless,  a  simple  visual  feedback  is  pro¬ 
vided  for  monitoring  and  software  verification  purposes  and,  due  to  the  modular  struc¬ 
ture  of  the  system,  binocular-visors  can  easily  be  interfaced.  It  should  be  mentioned  that 
considerable  risks  were  involved  in  realizing  the  tactile  display.  Since  an  effective  tech¬ 
nology  for  implementing  a  tactile  feedback  system  had  not  been  previously  established,  it 
was  rather  uncertain  whether  a  system  capable  of  providing  the  necessary  perceptual 
spatial  and  temporal  resolution  could  be  completed  within  the  limited  framework  of 
SCATIS.  The  project  was  carried  out  in  close  collaboration  with  Massimo  Bergamasco 
(Scuola  Superiore  S,  Anna,  Pisa,  I),  Dorte  Hammershpi  (University  of  Aalborg,  DK)  and 
the  industrial  partners  S.M.  Scienzia  Machinale  s.r.l,  Pisa,  I,  and  Head  acoustics  GmbH, 
Herzogenrath,  D. 

The  System ’s  General  Architecture 

The  architecture  of  the  SCATIS  VE-system  (the  so-called  SCAT-Lab)  was  designed  in  a 
hierarchical  and  modular  manner  so  as  to  provide  the  maximum  possible  flexibility  with 
regard  to  utilizing  hardware  and  extending  the  system  in  the  future.  The  modular  struc¬ 
ture,  which  is  depicted  in  Figure  1,  is  similar  to  the  architecture  proposed  by  Appino 
(Appino  et  al.,  1992).  All  the  important  components  of  the  SCAT-Lab  were  realized  as 
separate  UNIX  processes  which  can  be  deliberately  distributed  among  a  network  of 
workstations.  Communication  between  these  processes  is  provided  by  the  Central  Con¬ 
troller  (CC)  via  a  carefully  designed  number  of  messages  (events).  To  this  end  a  client- 
server  architecture  was  realized,  with  the  Central  Controller  being  the  only  client  and  all 


414 


other  modules  being  server  processes.  This  architecture  enables  the  Central  Controller  to 
control  the  timing  and  synchronization  of  the  connected  processes.  This  is  vital  for  a 
congruent  rendering  of  all  the  addressed  modalities.  The  client-server  behavior  is  explic¬ 
itly  forced  by  the  software  protocol.  The  position  and  orientation  of  the  head  and  hand 
are  frequently  measured  by  position  tracking  hardware,  based  on  modulated  electromag¬ 
netic  fields,  whereas  the  hand-finger  configuration  is  tracked  by  a  sensorized  glove 
equipped  with  kinesthetic  sensors  measuring  the  movements  of  the  proximal  and  medial 
joints  of  all  the  fingers  on  the  right  hand.  Measured  position,  orientation  and  angle  data 
acquired  by  these  two  devices  are  passed  on  to  the  Head&Hand  Tracking  Renderer 
(HHT)  process  that  computes  the  actual  position  and  orientation  of  each  individual  fin¬ 
ger-joint,  transforms  all  this  data  into  the  world  coordinate  system  and  buffers  the  data 
for  immediate  access  by  the  central  controller. 


Figure  1:  The  system  architecture  of  the  SCAT-Lab  virtual  environment 
generator. 

The  Central  Controller  (CC)  forms  the  link  between  the  efferent^  Head&Hand  Tracking 
Renderer,  the  Interaction  Module,  and  the  afferent  renderers  of  the  modal  pathways.  It 
accepts  movement  events  from  the  efferent  renderer  and  from  the  Interaction  Module, 
evaluates  them  according  to  the  rules  implemented  by  the  Application  and  reacts  by 
sending  appropriate  events  resulting  from  the  evaluation  to  the  afferent  renderers.  These 
rules  make  up  the  protocol  dynamics  of  a  virtual  environment  application.  For  example 
in  a  multi-modal  application  the  Central  Controller  could  advise  the  auditory  renderer  to 
switch  off  the  sound  of  a  sound  emitting  object  after  it  has  been  grasped  in  the  virtual 
world. 

The  purpose  of  the  Interaction  Module  (IM)  is  to  model  the  interaction  between  the  hand 
and  a  virtual  object.  This  task  includes  detecting  grasp  and  release  and  defining  how  a 
grasped  object  is  moved  when  the  hand  moves. 


'  Following  neurobiology,  modal  pathways  that  transfer  information  towards  the  subject  are  called  afferent  and  those  transfer¬ 
ring  information  from  the  subject  to  the  virtual  world  are  called  efferent. 
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The  task  of  the  afferent  tenderers  is  to  render  the  virtual  world  for  the  human  sense  they 
are  particularly  dedicated  to.  In  SCAHS  these  are  auditory  and  tactile  modalities.  The 
rendering  functions  are  distributed  among  several  modules  which  can  roughly  be  de¬ 
scribed  as  follows: 

1.  The  Tactile  Renderer  (TR)  detects  intersections  between  the  hand  and  other  active 
virtual  objects,  calculates  adequate  excitation  values  for  the  tactile  effectors,  and 
transmits  these  values  to  the  Tactile  Driver. 

2.  The  Tactile  Driver  (TD)  interfaces  the  Tactile  Renderer  with  the  tactile  effectors,  i.e., 
it  transforms  the  received  data  into  hardware-dependent  commands  and  implements 
the  communication  protocol  according  to  which  these  commands  are  transmitted  to 
the  tactile  effectors  via  a  serial  interface.  The  tactile  effector  consists  of  a  routing 
board  and  3  thermal-tactile  actuator-units.  Each  unit  incorporates  16  electro- 
magnetically  driven  indentor  pins  for  the  tactile  feedback  and  a  Peltier  cell  to  render 
thermal  feedback.  Emerging  waist  heat  is  drained  off  by  a  compressed-air  cooling 
system. 

3.  The  Auditory  Renderer  (AR)  performs  the  sound  field  modeling  by  calculating  the 
distribution  of  the  secondary  sound  sources.  This  depends  on  the  current  positions  and 
orientations  of  the  head  and  sound  sources  inside  the  virtual  environment.  It  deter¬ 
mines  the  delay  time,  the  directions  of  emittance  and  incidence,  and  the  affihated 
head-related  transfer  function  (HRTF)  for  each  direct  sound  source  and,  in  addition  to 
this,  the  wall  reflection  filters  for  each  early  reflection.  This  data  is  frequently  trans¬ 
mitted  to  the  Auditory  Driver, 

4.  The  task  of  the  Auditory  Driver  (AD)  is  to  auralize  the  various  sound  sources  and 
reflections  by  delaying  the  audio  input  signals  and  convolving  them  with  the  appro¬ 
priate  impulse  responses  representing  the  source  directivity,  the  wall  reflections  and 
HRTFs.  Finally  the  filtered  signals  are  added  together  and  equalized  for  the  transfer 
functions  of  the  headphones  employed.  From  this  the  binaural  output  signal  is  ob¬ 
tained.  Concerning  details  of  the  auditory  display  hardware  the  section  titled  „Real- 
Time  Auralization“  should  be  referred  to. 

5.  The  Visual  Renderer  (VR),  in  its  present  state,  is  merely  intended  to  provide  a  survey 
of  the  virtual  scenery  and  to  check  the  congruent  behavior  of  the  renderers  assigned  to 
the  different  modahties. 

Nevertheless,  as  a  consequence  of  the  modular  structure  of  the  VE-system  it  did  not 
prove  to  be  a  problem  either  to  integrate  additional  afferent  renderers,  e.g.  for  the  visual 
modality,  or  to  exclude  any  renderer  if  the  hardware  resources  are  insufficient  for  the 
complete  system  or  if  a  renderer  is  not  needed  for  a  certain  application. 

During  the  initialization  phase  each  renderer  sets  up  an  internal  database  by  parsing  a  file 
containing  a  complete  description  of  all  the  objects  that  make  up  the  virtual  world.  This 
database  may  be  customized  according  to  the  needs  of  the  particular  renderer.  The 
Auditory  Renderer,  for  instance,  is  very  interested  in  the  spatial  positions  of  the  head  and 
sound  emitting  or  -reflecting  objects,  whereas  the  configuration  of  the  hand  is  of  less 
interest.  Moreover,  each  afferent  renderer  keeps  additional  local  databases  containing 
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information  which  does  not  interfere  with  the  other  renderers’  domains.  The  common 
database  refers  to  these  local  databases  only  by  ids  of  their  entries  in  order  to  keep  the 
overall  memory  consumption  at  a  reasonable  level.  The  Tactile  Renderer,  e.g.,  maintains 
a  database  containing  the  haptic  attributes  of  the  surface  materials  involved  and  the 
Auditory  Renderer  needs  additional  information  on  the  directivity  of  sound  sources  and 
about  the  reflecting  properties  of  wall  materials. 

Two  UNIX  workstations  (16MFlops,  85Mips),  connected  via  Ethernet,  were  determined 
to  execute  all  the  processes  mentioned  above.  Optimal  utilization  of  the  CPU’s  capacities 
can  be  achieved  by  appropriately  distributing  the  processes  among  them.  If  necessary, 
additional  workstations  may  be  incorporated  into  the  simulation  to  increase  the  overall 
performance  of  the  VE-system. 

This  performance  can  partly  be  evaluated  using  the  two  characteristic  quantities  frame 
rate  (or  update  rate)  and  latent  period  (also  known  as  time-lag  or  latency  time).  Ac¬ 
cording  to  ANSI/IEEE  Std  100-1977,  the  latent  period  is  defined  as  the  time  elapsing 
between  the  application  of  a  stimulus  and  the  first  indication  of  a  response.  It  responds  to 
the  perceptual  attribute  responsiveness,  while  the  attribute  smoothness  can  be  assigned  to 
the  frame  rate  (Appino  et  al.,  1991).  The  frame  rate  is  the  average  number  of  discrete 
display  updates  that  can  be  rendered  per  time  interval.  One  objective  of  the  research  ac¬ 
tivities  with  the  SCAT-Lab  was  to  determine  the  perceptual  thresholds  of  these  quanti¬ 
ties. 

The  Auditory  Subsystem 

The  main  principles  of  binaural  room  simulation,  as  applied  in  SCATIS,  and  their  im¬ 
plementation  in  hard-  and  software  are  described  in  the  following  sections.  Please  refer  to 
(Lehnert  and  Blauert,  1989),  (Lehnert,  1992),  (Lehnert  and  Blauert,  1992),  and  (Strauss 
and  Blauert,  1995)  for  a  detailed  explanation  of  the  basics  of  binaural  room  simulation 
and  its  application  in  real-time.  Pompetzki  (Pompetzki  and  Blauert,  1994)  demonstrated 
the  perceptual  authenticity  of  binaural  room  simulation.  An  introduction  to  binaural 
technology  can  be  found  in  (Blauert,  1996)  and  an  exhaustive  overview  of  the  underlying 
psychophysics  of  human  sound  localization  is  given  by  Blauert  (Blauert,  1997). 

Binaural  Room  Simulation 

If  a  subject  is  to  be  placed  in  a  virtual  auditoiy  environment  it  is  necessary  to  present 
signals  at  both  eardrums  that  are  similar  to  the  signals  that  would  be  present  in  a  corre¬ 
sponding  real  environment.  Obviously  the  easiest  way  is  to  use  headphones  as  an  audi¬ 
tory  display,  because  the  binaural  signals  to  be  presented  to  the  eardrums  can  be  given 
directly  into  the  headphones’  transducer  terminals  after  having  adequately  equalized  for 
the  headphones’  transfer  function.  Disturbing  crosstalk  between  the  two  audio  channels 
cannot  occur.  The  tasks  to  enable  a  virtual  auditory  environment  to  be  created  are  as  fol¬ 
lows: 

1.  The  virtual  environment  must  be  modeled  from  the  acoustical  point  of  view  at  the 
listener’s  location.  This  is  called  auditory  rendering  or  sound  field  modeling, 

2.  Based  on  the  results  of  this  modeling  process,  the  resulting  eardrum  signals  have  to  be 
calculated  and  presented  which  is  known  as  auralization. 
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Auralizing  a  single  sound  source  is  comparatively  easy.  Given  the  position  and  orienta¬ 
tion  of  the  sound  source  and  of  the  subject’s  head,  the  distance  and  the  direction  of  inci¬ 
dence  can  be  calculated.  Auralization  is  performed  by  convolving  an  anechoic  sound 
signal  with  the  corresponding  HRIRs  in  real-time.  The  overall  gain  is  adjusted  according 
to  the  distance  between  the  sound  source  and  listener.  Absorption  of  sound  in  the  air  over 
long  distances  can  either  be  modeled  by  appropriate  overall  gain  reduction  or  by  fre¬ 
quency  dependent  gain  reduction.  Complex  directivity  characteristics  of  the  sound  source 
can  also  be  implemented  using  appropriate  pre-filtering  as  a  function  of  frequency  and 
direction  of  emittance.  Monopole  synthesis  and  spherical  harmonic  synthesis  have  been 
successfully  examined  for  the  purpose  of  efficiently  storing  directivity  data  (Giron, 
1993). 

The  model  as  yet  does  not  take  into  account  any  impact  of  the  surrounding  environment 
upon  the  listener’s  auditory  perception.  Many  properties  of  the  surrounding  environment 
can  be  obtained  solely  from  the  auditory  perception.  For  example,  the  interior  of  a  church 
sounds  completely  different  compared  to  a  small  living  room.  The  total  absence  of  re¬ 
flections,  for  example  in  an  anechoic  chamber,  can  even  be  an  unpleasant  sensation  for 
people  who  are  not  used  to  the  absent  auditorily  perceivable  environment.  Furthermore,  it 
is  believed  that  reflection  patterns  are  important  clues  to  proper  distance  perception. 

The  impact  of  the  reflective  environments  on  the  perceived  sound  can  be  modeled  with 
the  help  of  binaural  room  simulation  algorithms  (Lehnert,  1992).  These  algorithms  make 
use  of  geometric  acoustics:  Provided  the  wavelength  of  the  sound  is  short,  compared  to 
the  linear  geometric  dimensions  of  the  surfaces  in  the  room,  and  long,  compared  to  the 
roughness  and  curvatures  of  these  surfaces,  then  sound  waves  propagate  almost  in 
straight  lines.  They  can  therefore  be  treated  as  sound  rays  that  are  reflected  on  surfaces 
according  to  the  optical  reflection  law.  Though  the  assumption  above  is  not  true  for 
wavelengths  of  all  perceivable  sounds,  it  has  been  shown  (Pompetzki,  1993)  that  reason¬ 
able  results  can  be  achieved  using  geometric  acoustics.  Of  course,  wave  effects  like  dif¬ 
fraction  and  diffuse  reflections  cannot  be  exactly  modeled  using  geometric  acoustics  at 
first  hand.  This  would  require  the  acoustic  wave  equation  for  the  sound  pressure  p  to  be 
numerically  solved  with  respect  to  complex  boundary  conditions  given  by  the  reflective 
surfaces.  Since  a  great  deal  of  computational  effort  would  be  needed,  especially  for  high 
frequencies,  this  method  is  currently  not  suitable  for  real-time  applications. 

Based  on  the  above  simplification,  two  main  algorithms  have  been  developed  for  binau¬ 
ral  room  simulation.  According  to  the  so-called  mirror  image  model  (Allen  and  Berkley, 
1979),  primary  sound  sources  have  to  be  mirrored  on  all  geometric  surfaces  of  the  envi¬ 
ronment  to  obtain  virtual  secondary  sound  sources.  The  algorithm  can  be  applied  recur¬ 
sively  to  these  secondary  sources  to  obtain  results  for  higher  orders^.  However,  not  all  of 
the  secondary  sound  sources  found  with  this  procedure  are  acoustically  relevant  because 
the  surfaces  are  generally  not  extended  infinitely.  Therefore,  the  great  majority  of  the 
calculated  reflections  of  higher  orders  are  situated  outside  the  boundaries  of  the  corre¬ 
sponding  walls,  or  the  sound  path  is  blocked  by  other  walls.  Much  effort  in  terms  of  cal¬ 
culation  power  is  necessary  to  extract  the  relevant  sound  sources  by  performing  „visibil- 
ity“  investigations. 
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The  order  specifies  the  nun±»er  of  reflections  needed  for  the  sound  to  reach  the  listener. 


An  alternative  method  for  finding  secondary  sound  sources  is  the  Ray -Tracing  algo¬ 
rithm.  It  is  comparable  to  corresponding  rendering  algorithms  applied  in  the  field  of 
computer  graphics.  Rays  are  emitted  from  each  sound  source  in  different  directions  and 
their  propagation  inside  the  room  is  traced.  Rays  hitting  surfaces  of  the  environment  are 
reflected  according  to  the  reflection  law.  Diffuse  reflections  can  be  modeled  by  adding 
random  components  to  the  reflection  angle.  All  the  rays  that  hit  a  detection  sphere 
around  the  receiver  are  acoustically  relevant  for  the  simulation.  The  positions  of  secon¬ 
dary  sound  sources  can  easily  be  found  by  backtracking  these  rays.  However,  when  put¬ 
ting  the  Ray-Tracing  algoritlun  into  practice,  missing  or  multiple  detections  have  to  be 
considered  due  to  the  finite  number  of  rays  and  the  spatial  extension  of  the  detection 
sphere  around  the  receiver.  The  role  of  the  sound  source  and  the  receiver  can  also  be  re¬ 
versed.  This  requires  less  effort  if  more  than  one  sound  source  is  present. 

If,  in  theory,  calculation  time  were  unhmited,  both  algorithms  would  obtain  the  same 
distribution  of  virtual  sound  sources.  The  Ray-Tracing  algorithm  is  usually  more  effi¬ 
cient  for  finding  high  order  reflections  while  the  mirror  image  model  is  preferable  when 
only  low  order  reflections  are  required.  A  sequence  of  indices  is  assigned  to  each  secon¬ 
dary  sound  source  specifying  the  walls  where  the  sound  has  been  reflected  on  its  way  to 
the  listener.  Optionally,  reflection  angles  for  each  reflection  can  be  stored  for  simulating 
angle  dependent  reflection  characteristics. 

The  complete  characteristics  of  the  environment  as  a  result  of  this  sound  field  modeling 
process  can  therefore  be  represented  by  a  spatial  map  of  secondary  sound  sources 
(Lehnert  and  Blauert  1989)  in  a  reflection-free  space.  Each  sound  source  can  be  auralized 
with  the  help  of  previously  measured  head-related  transfer  functions  (cf.  e.g.  Pdsselt  et 
al,  1986  or  Hammersh0i  and  Sandvad  1994).  Directivity  characteristics  of  the  source 
and  wall  reflection  characteristics  can  be  modeled  using  pre-filters.  Angle-independent 
wall  reflectance  data  can  be  found  in  literature  or  obtained  from  direct  measurements 
usually  carried  out  in  reverberation  chambers.  The  result  of  this  primary  filter  process  is 
a  binaural  impulse  response  for  each  virtual  sound  source.  The  binaural  room  impulse 
response  is  obtained  by  adding  together  all  the  (appropriately  delayed)  binaural  impulse 
responses  from  each  virtual  sound  source.  It  can  be  interpreted  as  the  two  sound  pressure 
signals  at  the  eardrums  that  would  be  measured  if  the  sound  source  emitted  an  ideal  im¬ 
pulse,  The  room  impulse  response  completely  describes  the  transfer  characteristics  of  the 
environment  between  a  sound  source  and  the  listener.  For  auralization  purposes,  ane- 
choic  audio  signals  have  to  be  convolved  with  the  binaural  room  impulse  response. 

Real  Time  Auralization 

Auralizing  a  virtual  environment  by  convolving  the  audio  signals  assigned  to  the  sound 
sources  with  the  respective  binaural  room  impulse  responses  requires  an  enormous 
amount  of  calculation  power^  that  cannot  reasonably  be  performed  in  real  time,  so  sim¬ 
plifications  are  necessary.  Exact  auralization  can  be  restricted  to  first  and  second  order 
reflections  because  reflections  of  a  higher  order  can  hardly  be  perceived  separately. 
These  reflections  can  be  modeled  with  conventional  reverberation  algorithms  which  only 
consider  statistical  properties  of  the  late  reflections.  Direct  sound  and  first/second  order 


^  For  a  three  second  impulse  response  at  44,1  kHz  sampling  frequency,  nearly  raultiplications  per  second  and  channel  must  be 
calculated.  Using  the  overlap-add  algorithm  the  required  number  of  multiplications  may  be  significantly  reduced  at  the  cost  of  an  un¬ 
acceptable  additional  processing  delay. 
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reflections  can  be  auralized  in  real  time  by  distributing  the  corresponding  virtual  sound 
sources  among  a  network  of  several  digital  signal  processors  (DSPs)  where  each  virtual 
source  is  auralized  separately  by  convolving  the  corresponding  input  signal  with  the  re¬ 
spective  set  of  filters  and  delaying  it  appropriately.  In  the  framework  of  SCATIS  the  real 
time  auralization  was  realized  by  a  network  consisting  of  40  Motorola  DSP56002  signal 
processors  which  provide  an  overall  peak  calculation  power  of  800  million  multiplica¬ 
tions  per  second.  This  is  sufficient  for  auralizing  a  total  of  32  primary  sound  sources  and 
first  to  second  order  reflections  in  real  time  at  a  sampling  rate  of  48kHz. 


Figure  2:  Sketch  of  an  auralization  unit. 

The  system  for  auralizing  one  virtual  sound  source,  as  depicted  in  Figure  2,  is  called  an 
auralization  unit.  The  delay  is  proportional  to  the  travel  distance  between  the  source  and 
the  receiver  and  represents  the  time  the  sound  needs  to  reach  the  listener.  Pre-filter  1 
models  the  directivity  of  the  sound  source  and  is  selected  from  a  catalogue  according  to 
the  particular  direction  of  emittance.  The  remaining  two  pre-filters  represent  the  reflec- 
tional  properties  of  the  surfaces  the  sound  wave  is  reflected  at.  These  filters  may  not  only 
depend  on  the  frequency  but  also  on  the  angle  of  incidence.  Since  direction  dependent 
reflectance  data  is  presently  not  available  in  literature  and  extremely  difficult  to  obtain 
from  measurements,  a  catalogue  of  impulse  responses  of  a  great  variety  of  architecturally 
important  wall  materials  was  established.  At  present,  it  is  based  on  directionally  inde¬ 
pendent  absorption  data  supplied  by  the  German  Standards  Organization.  In  general,  all 
filters  are  implemented  as  FIR  filters  with  20  coefficients  for  each  pre-filter  whereas  the 
Head  Related  Impulse  Responses  (HRIRs)  (Sandvad  and  Hammersh0i,  1994)  employed 
for  the  spatiahzation  of  the  sound  sources  comprise  a  maximum  of  80  coefficients  per 
binaural  channel  (if  no  pre-filter  is  involved).  However,  since  the  convolution  power  of  a 
single  auralization  unit  is  limited  to  160  filter  taps  at  a  sampling  frequency  of  48  kHz, 
inserting  a  pre-filter  causes  a  cut  in  the  HRIRs  by  10  coefficients  per  channel.  This  may 
be  done  since,  as  far  as  reflections  of  higher  orders  are  concerned,  their  spectral  proper¬ 
ties  proved  to  be  more  important  than  those  referring  to  their  spatialization  (Lehnert, 
1995). 

The  three  pre-filters  as  employed  in  SCAT-Lab  allow  secondary  sound  sources  to  be 
auralized  up  to  the  second  order  including  directivity  characteristics  or  even  up  to  the 
third  order  if  directivity  characteristics  of  the  sound  source  are  not  simulated.  43  sets  of 
HRTFs  were  measured  with  38  different  subjects  with  a  resolution  of  11.25°  in  the  azi¬ 
muth  and  22.5°  in  the  elevation  angle,  each  set  consisting  of  195  transfer  functions  cov¬ 
ering  (nearly)  the  whole  sphere  from  -67.25°  to  +90°  elevation  (see  (Mpller  et  al.,  1995) 
for  details  of  the  measurement  method).  A  suitable  set  of  HRTFs  can  be  chosen  from  this 
catalogue  and  interpolated  off-line  by  the  auralization  hardware  in  order  to  increase  the 
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available  resolution  to  up  to  2°.  As  was  previously  mentioned,  all  the  auralization  units 
work  in  parallel  on  several  signal  processors,  their  results  are  added  together  and  finally 
equalized  for  the  employed  headphones  to  obtain  the  complete  binaural  output  signal. 

The  Demonstration  Scenarios 

Two  scenarios  were  implemented  to  demonstrate  and  test  the  capabilities  of  the  SCAT- 
Lab  auditory  auid  tactile  interfaces.  The  geometric  set-up  of  the  scenarios,  including  the 
initial  positions  of  the  virtual  objects,  was  defined  using  a  C-like  description  language, 
while  the  protocol  dynamics  were  coded  into  C-f-+  classes  derived  from  a  generic  appli¬ 
cation  class. 


Figure  3:  Sketch  of  the  virtual  studio  as  used  in  the  SCATIS  demonstra¬ 
tion  scenarios. 

Both  scenarios  are  located  in  a  small  to  medium  sized  virtual  studio  (about  6.4  m  x  5.3  m 
X  2,6  m,  volume:  35  m^)  with  a  simplified  mixing  console  inside  (see  Figure  3).  The  es¬ 
timated  reverberation  time  of  this  room  is  0.7  sec.  The  walls  of  the  room  cannot  be 
touched  by  the  subject.  Scenario  1  (“Virtual  Band“)  comprises  four  virtual  objects,  each 
of  them  representing  a  virtual  instrument  located  at  one  of  the  vertices  of  a  horizontal 
square  (2m*2m)  at  a  height  of  1.5m.  These  touchable  objects  with  simple  geometric 
shapes  are  defined  as  acoustically  transparent.  Initially,  only  one  of  them  emits  a  sound. 
The  task  of  the  subject  is  to  carry  out  a  certain  manipulative  procedure  on  this  object  so 
that  the  next  one  starts  to  emit  sound  as  well.  Again,  the  subject’s  task  is  to  manipulate 
the  newly  activated  object  in  order  to  activate  the  next  one.  The  task  continues  until  all 
four  objects  have  been  activated  and  the  fourth  one  has  been  treated  the  same  way.  Dur¬ 
ing  the  operation,  the  time  intervals  between  the  activation  of  the  different  objects  are 
measured.  These  time  intervals  are  expected  to  be  dependent  on  the  number  of  currently 
active  objects  and  also  on  the  positions  of  the  objects  in  space.  The  manipulative  proce¬ 
dure  the  subject  has  to  carry  out  on  an  object  in  order  to  activate  the  next  one  can  be 
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varied.  In  the  basic  scenario  this  manipulative  procedure  consists  of  just  a  grasping 
movement.  Possible  variations  are  to  turn  the  object  upside  down,  to  identify  a  particular 
side  of  the  object  with  differing  tactile  attributes  or  to  let  the  subject  move  the  activated 
objects  away  before  trying  to  grasp  the  next  one. 

The  second  scenario  employs  virtual  objects  with  different  haptic,  thermal  and/or  acous¬ 
tic  properties.  The  task  of  the  subject  is  to  auditorily  localize  these  objects  representing 
virtual  balls,  to  explore  their  properties  tactually,  and  to  move  them  into  special  areas  in 
the  subject’s  workspace,  which  represent  virtual  holes,  according  to  their  shape,  tem¬ 
perature  or  texture.  The  balls  emerge  one  by  one  at  different  spatial  positions  and  vanish 
again  as  soon  as  they  have  been  moved  to  die  correct  holes.  The  balls  and  holes  emit  two 
different  kinds  of  sounds  and  can  therefore  be  distinguished  auditorily  by  the  subject. 

Performance  Measurements 

Performance  tests  were  carried  out  measuring  the  frame  rate  and  the  latent  period  using 
the  test  scenarios  described  above.  To  this  end,  the  system  was  set  up  whereby  all  the 
afferent  renderers  were  initially  replaced  by  dummy  processes.  Then,  all  the  afferent  ren- 
derers  were  gradually  inserted  into  the  system.  In  principle,  the  frame  rate  was  limited  to 
60  Hz  by  the  maximum  speed  of  the  tracking  device  (Polhemus  Fastrak)  when  operating 
with  two  receivers.  The  processes  involved  in  the  measurements  were  executed  in  parallel 
and  in  changing  configurations  on  a  set  of  5  Silicon  Graphics  workstations  (2  Indigo 
R4000  +  3  Indy  R4600)  connected  via  Ethernet,  so  that,  in  general,  each  process  had  a 
CPU  of  its  own.  Applying  the  initial  concept  of  synchronous  rendering  (all  processes 
running  at  the  same  frame  rate)  a  frame  rate  of  60  Hz  could  be  achieved  in  any  configu¬ 
ration  of  processes  as  long  as  the  tactile  pathway  was  excluded  from  the  simulation  sys¬ 
tem.  The  overall  latent  period  of  the  auditory  pathway,  describing  the  elapsed  time  be¬ 
tween  the  measurement  of  the  head  position  and  the  adequate  audio  signal  being  dis¬ 
played  via  the  headphones  was  measured  to  approx.  80ms. 

As  soon  as  the  tactile  subsystem  (TR  -i-  TD,  both  running  on  the  same  host)  was  inte¬ 
grated  into  the  SCAT-Lab  the  frame  rate  dropped  drastically  to  about  12  Hz  at  maxi¬ 
mum,  although  the  load  of  none  of  the  CPUs  involved  exceeded  25%.  If  an  object  was 
touched,  the  frame  rate  dropped  even  further  to  about  10  Hz.  Since  the  integration  of  a 
visual  tenderer  running  on  the  same  host  together  with  the  CC,  the  TR,  and  the  TD  could 
not  even  decrease  the  overall  performance  significantly,  it  is  obvious  that  the  tactile  ef¬ 
fector  represented  the  limiting  factor.  The  latent  period  of  the  tactile  VE- system  turned 
out  to  be  within  the  range  of  300  ms  to  400  ms.  However,  despite  the  limited  overall  per¬ 
formance,  the  “Virtual  Band“  scenario  could  be  carried  out  successfully  in  a  systematic 
manner  by  several  subjects. 

Performance  tests  carried  out  with  all  the  rendering  processes  executing  asynchronously, 
revealed  that  the  overall  system  performance  can  be  increased  significantly  in  the  sense 
that  the  frame  rate  of  the  complete  system  is  no  longer  limited  by  its  slowest  component. 
The  performance  of  most  of  the  modules  is  increased  significantly  when  compared  to  the 
synchronous  rendering  strategy  that  was  initially  employed.  Since  the  performance  of  the 
tactile  subsystem  could  hardly  be  increased,  and  turned  out  to  be  independent  of  the 
workload  of  the  computer  it  was  executing  on,  it  can  be  assumed  that  the  bottleneck  is 
located  in  the  tactile  effector  hardware  or  in  the  serial  connection  to  the  computer  hosting 
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the  tactile  driver  process. 

Perceptual  Results 

Although  the  coherent  temporal  behavior  of  the  objects  in  both  modalities  is  mainly 
guaranteed  by  the  synchronous  rendering  strategy,  the  experimental  task  appeared  to  be 
quite  unnatural  and  even  tedious,  as  the  subject  is  obliged  to  move  her/his  hand  very 
slowly  in  order  to  maintain  the  spatial  constancy  of  the  virtual  environment.  If  the  head  is 
turned  rapidly,  the  sound  field  moves  perceivably  along  with  the  head  and  jumps  back 
into  its  former  spatial  position.  However,  this  proved  less  irritating  than  the  effect  caused 
by  the  tactile  display.  As  a  consequence  of  the  low  frame  rate  and  the  rather  long  latent 
period  that  was  experienced,  a  subject  has  to  approach  an  object  very  slowly  in  order  to 
detect  its  surface  with  high  spatial  precision.  Otherwise  the  spatial  and  temporal  resolu¬ 
tion  of  the  quantization  of  the  hand  and  finger  positions  is  insufficient  to  gain  perception 
of  contact  at  the  correct  spatial  position.  The  tactile  feedback  from  a  virtual  object  is  then 
usually  rendered  only  after  the  hand  has  already  entered  the  object.  In  this  case  the  tactile 
actuators  are  driven  to  full  (vibro-tactile)  output.  When  the  hand  is  moved  back  the  ex¬ 
citation  falls  back  to  zero  again,  delayed,  of  course,  by  the  latent  period.  This  kind  of 
,binary‘  behavior  makes  it  very  difficult  for  a  subject  to  obtain  a  proportional  tactile 
feedback  from  a  virtual  object’s  surface. 

Although  it  certainly  leads  to  larger  perceptible  inter-modal  temporal  discrepancies, 
asynchronous  rendering  proved  to  yield  a  much  better  perception  of  immersion  into  the 
simulated  scenario,  even  if  the  responsiveness  and  smoothness  for  one  of  the  addressed 
modalities  is  inadequate. 

Investigations  into  the  dynamic  aspects  of  auditory  virtual  environments  earned  by 
Sandvad  (1996)  showed  that  update  rates  below  20  Hz  and  latency  periods  of  96  ms  and 
above  have  a  significantly  degrading  effect  on  both  time  and  accuracy  of  sound  source 
localization  under  free-field  conditions.  These  results  which  correspond  with  tests  made 
by  the  authors  indicate  that  the  corresponding  values  achieved  with  the  auditory  compo¬ 
nent  (60  Hz  and  80  ms)  are  more  than  sufficient  to  obtain  a  plausible  representation  of 
an  auditory  virtual  environment. 

Conclusion 

The  experiments  show  that  the  main  aim  of  the  project  has  been  reached:  Subjects  ex¬ 
posed  to  the  auditory/tactile  environment  are  able  to  perform  the  experimental  tasks  as 
proposed  by  the  scenarios,  most  of  them  even  when  using  non-individualized  HRTFs. 
This  is  possible  even  at  the  very  limited  frame  rate  of  12  Hz  that  was  obtained  under  the 
generic  strategy  of  synchronous  communication  and  rendering. 

Since  the  latent  periods  of  the  auditory  and  the  tactile  VE  subsystems  turned  out  to  differ 
within  a  range  of  100  ms  to  300  ms,  even  when  applying  the  synchronous  rendering 
strategy,  temporal  congruence  of  the  two  modalities  cannot  really  be  guaranteed.  Apply¬ 
ing  asynchronous  rendering  strategies,  on  the  one  hand,  certainly  decreases  that  congru¬ 
ence  but,  on  the  other  hand,  it  significantly  enhances  the  smoothness  and  responsiveness 
of  at  least  the  auditory  domain  significantly  and  even  slightly  improves  the  tactile  do¬ 
main.  The  perceptual  effects  of  temporal  incongruent  rendering  for  the  two  modalities 
will  be  the  subject  of  future  research. 
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Considering  the  problems  detected  and  the  performance  figures  obtained  from  the 
SCAT-Lab  in  its  current  configuration,  the  applicability  for  multi-modal  research  ap¬ 
pears  to  be  still  limited.  However,  the  auditory  VE-technology  has  already  proved  its 
applicability  for  human  factors  research  and  industrial  use.  Convincing  results  in  terms 
of  plausible  real-time  simulations  of  rooms  with  moderately  complex  non-trivial  geome¬ 
try  can  be  obtained,  although  important  phenomena  like  the  dependence  of  the  HRTFs 
on  distances,  diffuse  reflections  at  walls,  diffraction,  and  Doppler- shifts  were  not  taken 
into  account  for  the  present  system.  These  effects  are  currently  subject  to  investigations 
and  will  be  partly  implemented  in  the  near  future. 

Although  the  tactile  pathway  works  in  principle,  its  performance  proved  to  be  greatly 
impeded  by  the  associated  display.  The  fast  and  efficient  rendering  algorithms  which 
have  been  developed  and  implemented  can  only  be  partly  transferred  to  create  adequate 
sensations  of  the  skin  because  the  present  capabilities  of  the  tactile  effectors  are  inade¬ 
quate.  The  tactile  system  needs  to  be  significantly  improved  especially  with  regard  to  its 
frame  rate  and  latent  period  in  order  to  facilitate  its  usage  in  advanced  VE-systems, 
However,  the  technological  approach  applied  appeared  to  be  quite  promising  for  future 
developments. 
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Abstract 

An  integrated  set  of  courses  for  a  track  of  four  subsequent  semesters  is  being  developed 
covering  computer  engineering  in  great  width  and  with  strong  links  to  technical  applica¬ 
tions.  The  courses  are  envisaged  to  become  integral  parts  of  a  distance  teaching  univer¬ 
sity’s  curriculum.  Combining  short  phases  of  presence  with  interleaved  media-based  self 
study  blocks,  they  are  also  well-suited  for  continuing  education  purposes.  An  innovative 
approach  is  taken  by  integrating  all  course  elements  -  acoustic  and  written  information, 
diagrams  and  figures,  animations,  simulations,  video  clips,  and  laboratory  exercises  -  in  a 
single  electronic  document.  Whereas  up  to  now  all  this  information  is  only  accessible  on 
different  storage  media,  the  new  multimedia  teaching  programmes  even  replace  printed 
material  to  a  large  extent. 

Introduction 

FemUniversitat  is  the  only  German  language  institution  of  higher  distance  education  at 
the  university  level,  i.e.,  granting  doctoral  degrees  and  habilitations.  It  has  students  in 
almost  all  countries  on  earth.  Engineering  curricula  are  offered  by  the  Faculties  of  Infor¬ 
matics  and  Electrical  Engineering  with  several  thousands  students  enrolled.  After  a  vo¬ 
cational  or  another  academic  training,  most  of  these  students  have  a  job  and,  in  parallel, 
are  further  educating  themselves  by  a  study  at  FemUniversitat.  They  are  motivated  by 
both  wishing  to  update  their  knowledge  and  to  enhance  their  career  opportunities  by  ob¬ 
taining  a  (higher)  academic  degree  (Diplom-Ingenieur,  Dr.-Ing.).  To  this  end,  they  either 
enroll  in  continuing  education  modules  consisting  of  a  number  of  courses  whose  success¬ 
ful  completion  is  certified,  or  by  following  an  entire  study.  In  the  latter  context  the  very 
successftil  supplementary  curriculum  in  electrical  engineering  is  to  be  particularly  men¬ 
tioned,  providing  engineers  having  graduated  at  Fachhochschulen  (polytechnics)  with  the 
opportunity  to  earn  an  university  diploma  in  a  shortened  study  period. 

Traditionally,  the  main  didactical  components  of  university  curricula  are  lectures,  exer¬ 
cises,  laboratory  practica,  and  seminars.  In  distance  teaching,  lecture  notes  and  corre¬ 
sponding  exercises  are  sent  to  the  students'  homes,  where  notes  are  studied  and  exercises 
elaborated  before  being  returned  for  marking  and  correction.  Hitherto,  self-study  course 
texts  and  exercises  are  shipped  in  form  of  printed  material  to  the  students,  who  return 
their  feedback  by  mail  as  well. 

At  FemUniversitat,  the  Faculty  of  Electrical  Engineering  is  transforming  all  its  courses  to 
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multimedia  form.  Thus,  they  can  be  distributed  on  CD-ROMs  emulating  traditional 
shipment  or  via  communication  networks  leaving  the  initiative  to  obtain  material  to  the 
students.  Electronic  mail  appears  to  be  the  most  appropriate  means  for  returning  exer¬ 
cises,  and  video  conferencing  is  successfully  employed  to  eliminate  seminar  travels. 

In  the  course  of  fundamentally  revising  the  contents  and  examination  regulations  of  Fer- 
nUniversitat's  electrical  engineering  curricula,  the  increased  requirements  of  electrical 
engineers  for  computing  knowledge  and  industry’s  need  for  demand-oriented  training 
were  particularly  taken  into  account.  According  to  this  need,  an  integrated  set  of  four 
subsequent  single  semester  courses  in  multimedia  form  was  defined  covering  computer 
engineering  in  great  width  and  with  strong  links  to  technical  applications.  The  courses 
will  become  integral  parts  of  the  mentioned  curricula  offered  by  FemUniversitat.  Com¬ 
bining  short  phases  of  presence  (e.g.,  weekend  seminars)  with  interleaved  media-based 
self  study  blocks,  they  are  also  well- suited  to  be  used  in  continuing  education. 

Project  Objectives  and  Considerations 

For  the  development  of  multimedia  courses  all  teaching  elements  -  acoustic  and  written 
information,  diagrams  and  figures,  animations,  simulations,  video  clips,  and  laboratory 
exercises  -  are  integrated  on  a  single  electronic  storage  medium  to  fully  exploit  the  possi¬ 
bilities  of  contemporary  teaching  and  learning  technology  and  to  lead  corresponding 
practices  far  into  a  new  area.  Phases  of  conveying  factual  knowledge  and  of  elaborating 
exercises  are  interleaved  by  integration  into  the  medium  computer  to  allow  for  a  maxi¬ 
mum  extent  of  interactive  exposure  to  the  teaching  material.  Combined  with  hypermedia 
techniques  complete  integration  offers  hitherto  unimagined  possibilities  to  dynamically 
adjust  teaching  material  to  the  students'  progress  of  learning  as  well  as  to  their  specific 
ways  of  learning.  Furthermore,  multimedia  enables  to  always  transfer  information  in  its 
most  suitable  form.  This  innovative  approach  is  not  without  risks  and,  thus,  has  to  be 
carefully  accompanied  with  methods  of  empirical  pedagogics. 

The  disadvantage  of  combined  media  introduced  several  decades  ago  was  that  their  sin¬ 
gle  elements  could  only  be  made  accessible  on  different  storage  media  (study  texts,  audio 
cassettes,  movies,  video  tapes,  video  disks,  diskettes)  each  -  except  the  texts  -  requiring  a 
special  device  for  replay.  Therefore,  it  was  practically  difficult  to  have  the  complete  vol¬ 
ume  of  information  at  one's  disposal,  and  its  manageability  was  particularly  cumber¬ 
some.  This  made  it  advisable  to  keep  the  interleaving  of  the  information  offered  on  the 
different  media  rather  limited.  The  older  multimedia  concepts  tried  to  overcome  this 
shortcoming,  at  least  a  little  bit,  by  "programming”  the  linkage  of  the  still  separated  me¬ 
dia  and,  thus,  realising  an  integrated  sequence  control. 

Our  approach  takes  a  decisive  step  ahead:  information  (text,  sound,  picture,  movie  etc.) 
hitherto  separated  by  different  media  is  integrated  on  a  single,  qualitatively  as  well  as 
quantitatively  high  performance  storage  medium.  Complete  availability  of  all  informa¬ 
tion  is  always  ensured,  and  in  combination  with  hypermedia  techniques  high  flexibility  is 
given  for  accessing  different  information.  Furthermore,  multimedia  offer  a  great  potential 
for  technology  migration,  since  information  can  always  be  communicated  in  the  most 
suitable  form. 

Certainly  it  makes  no  sense  to  offer  large  quantities  of  written  information  for  perception 
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via  a  computer  screen.  Therefore,  in  general  FernUniversitat’s  learning  programmes  do 
not  present  larger  quantities  of  text.  This  is  achieved  by  either  selecting  for  programmes 
such  topics  which  require  only  little  text,  or  by  letting  programmes  accompany  printed 
course  material,  or  by  adding  supplementary  written  information  to  programmes.  Hith¬ 
erto,  teaching  software  mainly  concentrated  on  functions  which  cannot  or  which  can  only 
inappropriately  be  fulfilled  by  written  courses,  such  as  to  present  and  deal  with  processes 
of  all  kinds  or  to  support  active  perception  of  the  material.  However,  after  it  is  now  tech¬ 
nically  possible  to  integrate  acoustic  information  including  spoken  texts  into  teaching 
programmes,  the  latter  cannot  only  accompany  and  supplement  printed  courses  but,  in 
principle,  also  replace  them.  Thus,  electronic  courses  become  possible  as  integrated 
multimedia  programmes.  The  most  striking  characteristic  of  our  approach  is  to  overcome 
combined  media  by  integration  and  completely  relinquishing  printed  media. 

Teaching  media  designed  as  self-instructing  materials  -  this  is  the  case  for  all  distance 
education  materials  -  can,  in  principle,  be  used  in  three  forms  of  teaching  and  study  or¬ 
ganisation:  as  studying  materials  at  an  institution  of  higher  distance  education,  as  study¬ 
ing  materials  in  a  single  phase  of  distance  studies  organised  by  an  institution  of  presence 
education  or  by  a  distance  learning  institution  in  co-operation  with  an  institution  of  pres¬ 
ence  learning,  or  as  self- studying  materials  at  institutions  of  presence  education  in  order 
to  supplement  or  to  replace  lectures.  The  multimedia  learning  programme  "Computer 
Engineering"  is  designed  to  be  employed  (entirely  or  in  defined  parts)  in  all  these  areas. 

Didactic  Features 

The  project’s  most  striking  characteristic  is  overcoming  combined  media  by  real,  physi¬ 
cal  and  complete  integration  and  the  total  renunciation  of  printed  media.  This  approach 
contains  considerable  potential  for  innovation.  It  requires  to  further  develop  didactic- 
methodical  concepts  of  distance  teaching,  and  of  media-based  learning  in  general.  Thus, 
the  main  question  is:  with  which  didactic-methodical  concept  and  with  which  practical 
implementation  technique  can  a  modem  electronic  course  be  developed  that  gets  by 
without  printed  media  and  combines  different  information  elements  such  as  text,  sound, 
picture,  and  movie,  that  are  hitherto  stored  on  separate  media,  in  a  single  electronic 
document? 

In  particular,  the  didactic-methodical  concept  has  to  contain  suggestions,  how  the  differ¬ 
ent  elements,  mainly  acoustic  and  written  information,  can  be  optimally  co-ordinated  for 
the  presentation  of  the  material  itself  and  of  exercises  and  problem  sets.  The  concept's 
realisation  and  its  validation  are  carried  out  step  by  step  (formative  evaluation).  It  is 
hoped  that  the  optimum  co-ordination  will  have  been  found  by  the  end  of  the  project. 

The  electronic  courses  are  conceived  in  such  a  way  that  they  may  be  used  not  only  for 
distance  learning,  but  also  as  self-studying  materials.  To  this  end,  the  material  is  struc¬ 
tured  in  modules.  In  the  course  of  the  project  it  will  become  clear,  whether  this  goal  can 
be  achieved. 

In  pursuing  the  objectives  stated  above,  it  is  looked  into  the  question  how  teaching  of  a 
quite  technical  subject  with  corresponding  laboratory  phases  can  be  interleaved  by  inte¬ 
gration  in  the  medium  computer.  How  to  explain  and  to  accustom  students  with  the 
function  of  a  physical  device  when  there  are  no  laboratory  sessions?  As  now  common  in 
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science,  it  is  not  dealt  with  the  real  world,  but  components  up  to  computer  cores  are 
simulated.  Acceptance  and  range  of  this  virtual  world  are  to  be  investigated  in  the  par¬ 
ticular  subject  area  of  computer  engineering.  Basic  digital  circuits  and  electronic  gates, 
for  instance,  are  represented  graphically  in  diagrams  -  the  input  values  can  be  assessed 
by  students.  Simulation  programs  then  provide  the  same  values  at  the  outputs  like  in  a 
real  circuit. 


A  central  item  of  the  courses  is  a  simulation  program  for  an  easily  programmable  digital 
computer.  This  hypothetical  computer  does  not  match  with  any  really  existing  computer. 
On  the  one  hand,  it  is  so  powerful  that  it  possesses  the  most  important  properties  of 
digital  computers  dealt  with  in  the  course.  On  the  other  hand,  it  is  so  simple  that  it  dis¬ 
penses  with  all  further  special  features  that  might  be  disturbing  while  demonstrating  the 
basic  properties  of  a  computer.  As  depicted  in  Figure  1,  this  computer  shows  the  contents 
of  its  memory  and  the  most  important  registers  on  its  graphical  user  interface,  and  is  pro¬ 
grammable  with  the  help  of  a  set  of  20  instructions.  The  execution  of  a  program  can  be 
performed  either  in  step  mode  or  in  free  running  mode,  as  desired. 
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Fig.  1:  Simulation  of  a  computer  core 


With  these  simulated  circuits  and  computers,  first  the  mode  of  operation  of  systems  can 
be  demonstrated  with  examples,  especially  the  basic  working  method  of  a  digital  com¬ 
puter  and  its  programming.  Furthermore,  students  are  given  the  opportunity  to  work  ac¬ 
tively  on  the  problems  dealt  with  in  the  course  and  to  develop  practical  experiences  in  the 
handling  of  systems.  Finally,  the  simulated  devices  are  at  disposal  for  free  experimenting, 
which  might  lead  to  new  questions  and  the  search  for  own  answers. 
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In  the  project  it  is  investigated,  whether  the  need  of  practical  experiences  in  the  subject 
of  computer  engineering  can  be  completely  covered  in  the  way  described,  in  how  far  this 
offer  is  accepted,  and  how  it  is  assessed. 

Another  question  concerning  the  complete  integration  and  the  relinquishment  of  wntten 
media  arises  because  of  the  commonly  held  estimation  that  it  is  not  feasible  to  offer  ex¬ 
tensive  written  information  for  reading  on  a  monitor.  Admittedly,  longer  verbal  passages 
usually  do  not  take  up  a  central  position  in  the  passing  on  of  technical  contents,  but  they 
do  have  their  place  in  the  field  of  computer  engineering,  e.g.,  in  introductions  into  a 
subject,  in  motivations  for  a  problem  or  in  supplemental^  remarks  about  the  reference  to 
practice  or  the  like.  But  since  the  integration  of  acoustic  information  is  now  technically 
realisable,  it  is  possible  -  and  in  the  sense  of  easing  the  visual  channel  even  desirable  -  to 
offer  such  information  as  spoken  texts.  A  typical  example  for  this  type  of  information  in 
the  programme  "Computer  Engineering"  is  made  up  by  a  historical  look  back  at  the  de¬ 
velopment  of  calculators,  hi  the  style  of  a  sound-slide- show,  a  detailed  spoken  descrip¬ 
tion  of  the  development  is  given  together  v^dth  several  pictures  of  inventors  and  their 
historical  machines. 
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Fig,  2:  Demonstration  of  an  algorithm 

The  concept  of  the  integrated  information  is  in  essential  parts  superior  to  conventional 
presentations  in  study  texts,  lectures  or  films.  As  there  is  no  longer  a  limitation  to  the 
respectively  small  presentation  facilities,  it  is  now  possible  to  present  any  content  in  its 
optimal  form.  As  examples  that  are  of  special  importance  in  the  field  of  computer  engi- 
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neering  the  picture-sound-sequences  are  to  be  cited,  well-known  from  educational  films, 
i.e„  sequences  of  diagrams,  formulae  etc.  that  are  accompanied  by  sound  commentaries. 
Thus,  gradual  developments  and  processes,  like  e.g,,  the  construction  of  a  complex  dia¬ 
gram,  the  derivation  of  a  mathematical  formula  or  -  especially  in  computer  engineering  - 
the  demonstration  of  the  operation  mode  of  algorithms  or  circuits  as  shown  in  Figures  2 
and  3,  respectively,  can  be  presented  graphically  by  picture  sequences,  and  -  in  contrast 
to  written  material  -  explained  at  the  same  time  in  an  optimal  way.  Most  importantly,  and 
in  contrast  to  educational  films,  the  speed  of  the  process  can  be  determined  by  the  user 
himself. 

Of  even  greater  importance  as  for  the  presentation  of  the  subject  are  the  aspects  of  new 
presentation  and  individual  usage  facilities  in  the  field  of  the  use  of  computer  programs. 
With  the  help  of  simulated  circuits  and  devices,  now  -  conventionally  only  in  the  context 
of  a  special  practical  instruction  -  the  gaining  of  practical  experiences  is  possible.  The 
passing  on  of  the  syllabus  and  the  practical  exercises  can  be  interconnected  in  a  way  cho¬ 
sen  by  the  student  himself. 

Completely  new  is  also  the  approach  to  provide  the  student  with  an  interactive  environ¬ 
ment  of  software  tools.  This  surrounding  offers  the  opportunity  to  deepen  the  under¬ 
standing  of  the  learned  terms  by  means  of  trial  and  error.  Thus,  e.g.,  the  working  method 
of  circuit  networks,  digital  circuitiy,  finite  automata,  algorithms  or  complete  computer 
architectures  can  be  illustrated  by  animation,  interaction,  colour  and  sound  accompani¬ 
ment.  A  further  feature  of  the  software  environment  is  the  opportunity  to  adapt  and  ex¬ 
pand  the  functionality  of  the  tools  in  correspondence  to  the  user’s  learning  process. 
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3.2.2  Entwutf von  Gatterschaltungen 


Satz 

Jedes  System  Boolescher 
Funktionen  laBt  sich 

-  durch  eine  Gatterschaltung  in 
NICHT-UND-ODER-Technologie, 

-  dutch  eine  Gatterschaltung  in 
nand-T  echnologie  und 

-  durch  eine  Gatterschaltung  in 
NOR-Technologie 

realisieren. 


. . — . . . IV 


Fig,  3:  Demonstration  of  a  circuit 
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In  addition  to  the  advantages  of  the  concepts  of  integrated  information,  that  are  to  be 
found  in  representing  and  operating  the  single  information  elements,  there  is  a  further 
crucial  advantage  that  is  based  on  the  overall  structure  of  the  teaching  programme:  a 
computer  teaching  programme  is  able  to  overcome  the  presentation  that  is  ordered  line¬ 
arly  and  to  be  passed  through  sequentially,  as  it  is  given  in  books  or  study  texts,  lectures 
and  films.  With  such  a  suitable  structure,  on  the  one  hand  the  individual  student  can  be 
provided  with  essentially  better  opportunities  to  go  through  the  subject  in  a  way  that  is  in 
accordance  with  his  individual  background  knowledge  and  learning  experiences,  on  the 
other  hand  a  large,  even  quite  heterogeneous  range  of  persons  can  be  addressed. 

The  students  that  are  addressed  by  the  teaching  programme  "Computer  Engineering" 
have  the  most  different  background  knowledge  and  learning  experiences.  This  is  valid 
for  the  students  of  FemUniversitat,  among  whom  there  are  students  with  modest  school 
knowledge,  practitioners  with  the  most  different  abilities,  and  university  graduates  of  a 
related  field,  and  it  is  even  more  valid,  if  a  further  range  of  students,  e.g.,  at  other  univer¬ 
sities  or  in  the  field  of  continuing  education,  is  addressed. 

The  requirements  for  teaching  material  to  be  suitable  for  such  a  wide  range  of  addressees 
and,  at  the  same  time,  to  be  effectively  manageable  by  the  individual  student  are  only 
very  hard  to  meet  by  subjects  presented  in  a  linear  fashion  (book,  lecture,  video  clips). 
This  is  due  to  the  fact  that  clearly  and  stringently  presented  material  can  only  be  compre¬ 
hended  in  an  optimal  way  by  learners  with  certain  background  knowledge  and  learning 
abilities.  Material  endowed  with  learning  aids,  however,  is  confusing  and  only  ponder¬ 
ously  manageable  by  the  individual  person. 

A  computer  program,  however,  offers  the  chance  to  provide  an  amount  of  learning  aids 
and,  furthermore,  to  structure  this  offer  in  such  a  way  that  the  learner  effectively  can 
make  his  choice  that  is  in  accordance  with  his  individual  needs.  It  is  our  approach  to 
make  a  strict  distinction  between  the  contents  to  be  passed  on  -  e.g,,  the  definitions, 
propositions,  methods,  algorithms  or  descriptions  of  devices  or  circuits  -  that  are  com¬ 
pulsory  for  everyone,  and  the  didactic  elements  -  introductions  into  a  subject,  motiva¬ 
tions,  explanations,  examples,  demonstrations,  exercises,  lists  of  teaching  aims,  provision 
of  background  knowledge  from  mathematics  and  electrical  engineering  -  that  are  only 
offered  optionally. 

In  the  programme  "Computer  Engineering",  the  contents  to  be  passed  on  are  decomposed 
in  small  sections  that  are  offered  in  linear  order.  To  each  of  these  content  sections  a  set  of 
didactic  elements  is  allocated  that  can  be  chosen  as  required  via  a  selection  menu.  Thus, 
e.g.,  a  student  repeating  the  course  will  mainly  go  through  the  content  elements  before  an 
examination  and  only  rarely  take  advantage  of  supplementary  information,  whereas  a 
beginner  in  this  subject  possibly  falls  back  on  most  of  the  aids.  Thus,  an  offer  arises  that 
on  the  one  hand  copes  with  the  diverse  needs  of  learning  support,  and  on  the  other  hand 
is  simply  and  clearly  to  handle  and  -  especially  with  the  help  of  the  lists  of  teaching  aims 
and  self-test  exercises  offered  in  every  section  -  enables  an  effective  and  object-oriented 
working  on  the  syllabus. 

Course  Development 

The  project  work  consists  of  development  activities  that  are  directly  followed  by  testing 
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activities.  The  former  can  be  subdivided  into  teaching  (selection  and  restriction  of  mate¬ 
rial,  definition  of  pre-conditions  and  expected  performance,  standards  of  grading,  etc.) 
and  presentation  activities  (didactics  of  the  media,  design  of  teaching  media,  technology 
of  teaching  and  learning,  etc.).  As  far  as  the  testing  activities  are  concerned,  we  distin¬ 
guish  between  (further)  teaching  (organisation  of  the  situation  of  teaching  and  learning, 
selection  of  parts  to  be  tested,  assessment  of  the  evaluation  results,  etc.)  and  research 
activities  (formulation  of  hypotheses  and  questions,  selection  of  testing  instruments,  as¬ 
sessment  of  testing  results,  etc.). 

The  sequencing  of  development  and  testing  phases  characterises  the  project  s  schedule. 

1 .  development  of  the  pilot  unit, 

2.  testing  of  the  pilot  unit, 

3.  development  of  the  first  half  and  important  parts  of  the  second  half  of  the  course 
package, 

4.  testing  of  the  first  half  of  the  course  package  and  further  development  of  the  second 
half  of  the  four  courses, 

5.  testing  of  the  second  half  of  the  course  package  and  adding  testing  results,  and  finally 

6.  adding  of  testing  results  and  completion  of  the  course  package. 

It  is  estimated  that  some  150  "person-months”  are  required  for  the  complete  development 
of  the  course  package. 

Course  Assessment 

Empirical  pedagogics  distinguishes  between  formative  and  summative  evaluation. 
Whereas  the  latter  investigates  the  prepared  course,  i.e.,  the  developed  product,  the  for¬ 
mer  focuses  on  the  development  process.  As  in  the  present  project,  formative  evaluation 
is  always  selected  when  a  new  type  of  a  teaching  medium  is  to  be  developed.  Therefore, 
testing  phases  are  scheduled  within  the  development  period  as  formative  evaluation.  This 
will  concentrate  on  structure,  contents,  and  methodics  of  the  course  package.  The  main 
questions  aim  at  clarity  and  consistency  of  the  course  structure,  the  optimum  proportion 
of  informative,  activating,  and  testing  contents,  the  optimal  balance  of  sound,  text,  pic¬ 
tures,  films,  and  simulations,  as  well  as  whether  the  teaching  programme  is  reliable  and 
easy  to  handle.  Needless  to  say,  formative  evaluation  is  to  detect  and  remove  errors. 

The  schedule  provides  three  phases  of  formative  evaluation,  with  the  first  one  serving  for 
the  better  formulation  of  hypotheses  and  questions,  whereas  the  second  and  third  phases 
are  totally  aimed  to  gain  more  knowledge.  The  first  phase  will  be  carried  out  using  the 
pilot  unit  with  just  a  few  students  in  class  at  FemUniversitat.  Owing  to  the  objectives  and 
the  low  number  of  participants,  in  this  phase  it  is  tried  to  obtain  first  results  by  observa¬ 
tions  and  (group)  discussions.  It  is  not  sensible  to  use  interviews  and  questionnaires  be¬ 
fore  the  second  and  third  evaluation  phase,  following  the  first  one  with  a  distinct  delay. 
The  evaluation  sites  will  partly  move  to  the  students’  homes,  in  order  to  meet  the  condi¬ 
tions  of  distance  learning.  It  is  envisaged  to  include  up  to  50  participants  into  these  tests. 
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They  will  be  questioned  following  a  structured  programme. 

Distance  Laboratory 

Whereas  the  utilisation  of  new  technology  for  the  above  mentioned  purposes  in  distance 
education  is  straightforward,  it  is  not  so  obvious  to  find  under  severe  cost  limitations  new 
ways  of  allowing  students  to  conduct  -  from  their  homes  -  laboratory  projects  with  ex¬ 
perimental  work,  indispensable  for  the  understanding  of  technical  subjects. 

Therefore,  up  to  now,  two  approaches  are  taken.  One  possibility  is  to  arrange  practica  in 
laboratories  on-site  a  university,  that  require  the  local  presence  of  the  students.  In  order 
to  save  time  and  travel  costs,  it  is  essential  to  schedule  these  practica  close  together  and 
to  minimise  their  number.  The  other  approach  is  to  carry  out  experiments  at  home.  To 
this  end,  FemUniversitat  sends  out  experimentation  kits  such  as  microprocessor  boards. 
Their  disadvantage  is  that  they  have  to  be  returned  together  with  the  results  obtained. 
This  method  has  many  drawbacks  such  as  limited  kit  size,  damages  during  shipment,  and 
high  costs  for  inspection,  repair,  replacement,  and  handling. 

The  above  considerations  establish  the  need  for  a  new  approach  to  conduct  technical 
experimentation  under  the  conditions  of  distance  teaching.  The  requirements  to  be  ful¬ 
filled  by  a  new  system  are  reasonable  costs,  no  need  for  travel,  and  provision  of  a  generic 
environment  allowing  for  great  flexibility  in  experimentation. 

A  solution  developed  to  meet  this  need  is  called  Distance  Laboratory,  Since  actually  all 
students  have  access  to  a  personal  computer,  which  can  be  linked  to  the  telephone  net¬ 
work  with  a  modem,  it  does  not  require  any  new  investments  on  the  students’  side.  For 
experimentation,  the  students  work  with  particular  programs  on  their  computers.  In  the 
course  of  this,  data  are  generated  and  transmitted  via  a  network  to  FemUniversitat,  where 
they  are  loaded  into  automatically  operated  experimental  rigs.  Results  are  returned 
showing  the  students  in  appropriate  ways  how  their  experiments  ran  and  providing  feed¬ 
back  for  improvements  or  corrections  in  case  of  errors. 

In  more  detail,  on  their  personal  computers  at  home,  with  tools  provided  by  FemUniver¬ 
sitat  the  students  develop  control  software  for  their  experiments,  which  are  first  simu¬ 
lated.  When  this  step  is  successfully  completed,  the  control  software  is  transmitted  via 
the  Internet  using,  for  instance,  the  electronic  mail  service  to  a  host  at  FemUniversitat. 
The  host  shown  in  Figure  4  queues  the  requests,  simulates  the  experimental  mns  once 
more  for  safety  purposes  and,  then,  down-loads  each  control  program  to  a  corresponding 
rig  for  canying  out  an  experiment.  During  the  experimental  mn  the  host  samples  status 
data,  which  are  processed  and  returned  to  the  student  to  provide  information  on  the  out¬ 
come  of  the  experiment. 
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Fig.  4:  Configuration  of  Distance  Laboratory 


An  exercise  on  the  design  of  digital  circuits,  for  instance,  can  be  carried  out  via  a  net¬ 
work  in  its  entirety.  It  comprises  the  design  of  a  digital  circuit,  its  implementation  using 
programmable  logic,  and  also  the  test  of  configured,  real  hardware.  There  are  further  ap¬ 
plications  of  this  concept  in  robotics,  programmable  logic  control,  and  distributed  digital 
control. 
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Abstract 

Construction  of  electronic  hardware  is  done  by  engineers  which  need  a  real  good 
education  during  their  studies.  The  approach  we  are  describing  below  has  proven  a  good 
means  for  performing  this  difficult  task.  The  transfer  of  synthesis  knowledge  is  a  task 
which  covers  theoretical  as  well  as  practical  aspects.  This  usually  leads  to  courses  which 
are  split  in  two  parts,  the  lecture  part  and  the  practical  training  part.  The  concept 
described  here  shows  a  way  how  to  cope  with  this  problem.  In  addition,  the  internet  as 
information  carrier  opens  quite  new  ways  of  interaction  between  the  teacher  and  the 
course  participant.  The  participant  has  access  to  all  course  materials  and  he  can  use  the 
tools  via  the  internet.  The  strong  binding  to  a  particular  location,  where  the  high  end 
workstations  and  the  commercial  software  are  available  is  resolved,  too.  Finally,  the 
availability  on  the  net  makes  the  system  suited  for  teaching  at  the  university  as  well  as  for 
training  within  a  company. 

Introduction 

Teaching  in  areas  where  new  technologies  have  a  strong  influence  requires  a  steady 
update  of  the  lecture  contents.  In  addition,  the  tools  used  for  the  design  of  digital  systems 
like  processors  become  more  and  more  complex  and  are  changing  continuously,  which 
makes  the  update  process  for  the  course  materials  look  like  the  fight  of  Don  Quichote 
against  the  windmills.  On  the  other  hand,  nobody  is  really  able  to  spend  all  his  time  in  the 
preparation  of  new  lecture  materials.  This  contradiction  must  be  resolved  if  we  want  to 
keep  pace  with  the  technical  evolution  in  education. 

The  way  how  we  faced  this  challenge  was  to  start  a  cooperation  with  the  idea  in  mind  to 
apply  the  well  known  concept  of  divide  and  conquer  to  the  area  of  teaching  synthesis  of 
digital  system.  Here  dividing  means  that  each  participant  focuses  on  the  area  he  knows 
best  so  that  he  does  not  have  to  cope  with  the  complete  area.  Conquering  means,  that  we 
share  our  contributions  freely  so  that  every  participant  can  make  use  of  all  available 
information  and  tailor  it  to  his  own  needs.  The  medium  we  have  chosen  for  the 
implementation  of  the  concept  is  the  internet.  The  growing  information  pool  is  not 
printed  and  mailed,  but  it  is  made  available  as  HTML^  text  which  reduces  the  effort  in 
maintaining  the  data  enormously. 

Finally,  this  approach  offers  new  ways  of  cooperation  between  teacher  and  student.  Pre¬ 
viously  the  teacher  used  a  set  of  slides  and  recommended  one  or  more  textbooks.  Here 
the  global  availability  gives  much  more  freedom  to  the  student  to  select  material  which 
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may  have  been  presented  by  the  teacher  or  is  part  of  the  overall  information  pool.  In 
addition,  the  new  medium  allows  the  integration  of  text,  graphics,  animations  and  real 
tools  into  one  homogenous  environment.  In  this  paper  we  will  first  introduce  the  course 
structure  to  illustrate  how  we  are  teaching  system  design.  In  the  second  part  we  will  show 
all  the  various  means  we  are  using,  like  www  pages,  animations,  tool  interfaces  etc. 

Courses  on  synthesis  of  digital  systems 

Digital  system  design  today  means  that  the  behaviour  of  a  system  is  described  in  a  kind 
of  programming  language,  describing  the  behaviour  similar  to  a  standard  language,  but 
introducting  mechanisms  for  structured  designs,  too.  Timing  information,  which  usually 
is  not  part  of  a  programming  language  becomes  a  key  issue.  During  the  synthesis  process 
the  abstract  description  is  the  mapped  to  real  gates  and  cells  which  can  finally  be 
implemented  in  silicon. 

Any  course  on  digital  system  design  has  to  cope  with  several  aspects  of  teaching,  each  re¬ 
quiring  different  approaches.  First,  you  have  to  teach  basic  technologies  so  that  the 
students  get  an  understanding  about  all  the  constraints  which  have  to  be  kept  in  mind 
during  the  design  process.  Then  the  students  need  to  know  about  the  design  process  and 
design  representations,  beginning  with  simple  logic  gates  and  ending  with  behavioural 
descriptions.  Topics  like  simulation,  test  and  testability  have  to  be  covered,  too.  Finally, 
they  have  to  be  trained  in  the  use  of  commercial  design  systems,  because  otherwise  the 
procedure  is  like  learning  a  programming  language  by  writing  code  on  a  sheet  of  paper. 
Luckily  parts  of  this  highly  complex  program  is  taught  in  other  courses,  so  we  do  not 
have  to  cover  everything.  Still,  the  amount  of  information  to  be  transferred  is  very  big.  So 
the  teacher  has  to  select  some  parts  of  it  and  create  a  curriculum  which  fits  into  the 
overall  course  structure  of  the  faculty  as  well  as  to  his  own  ideas. 

As  an  example  we  will  outline  the  design  course  at  Siegen  University.  It  is  called  Entwurf 
Integrierter  Systeme  (E.LS,,  pronounced  ice  =  design  of  integrated  systems).  It  covers 
two  semesters.  During  the  first  semester  the  students  learn  about  full  custom  design  and 
semi  custom  design.  The  lecture  part  begins  with  all  what  is  needed  for  the  design  of  a 
simple  cell,  which  is  e.g.  the  function  of  a  CMOS  inverter,  the  influence  of  fan  out  and 
fan  in  and  timing  definitions.  The  model  which  is  required  for  the  simulation  of  such  a 
cell  is  part  of  the  lecture,  too.  Later  on,  the  knowledge  about  the  design  of  a  digital  logic 
is  shortly  refreshed.  Test  methods  and  approaches  to  design  for  testability  are  introduced. 
The  lecture  completes  describing  the  functionality  of  a  digital  simulator,  test  pattern 
generation  and  fault  simulation.  In  parallel,  the  students  design  a  single  cell  using  the 
Cadence  system  and  simulate  it  with  Hspice.  Then  a  small  standard  cell  design  is  made  to 
practice  this  method  and  learn  about  the  design  problems  an  this  level.  Usually  the 
students  start  a  competition  which  group  gets  the  fastest  design.  The  teaching  concept  for 
this  part  is  currently  still  conservative  which  means  that  we  use  conventional  slides. 

The  second  semester  focuses  on  system  level  specification,  using  VHDL^  as  the  basic 
hardware  description  language  and  the  synthesis  tools  from  Synopsys.  The  system  we  are 
describing  in  this  paper  covers  the  second  semester  of  this  course.  The  lecture  part 
consists  of  teaching  various  system  description  methodologies.  It  starts  with  abstract 
approaches  like  Petri  Nets  and  state  charts.  Emphasis  is  set  on  behaviour  descriptions. 
The  next  step  covers  scheduling  and  allocation.  The  synthesis  steps  are  explained  and 
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FPGAs  are  introduced  as  an  example  technology  which  permits  rapid  prototyping  and  for 
the  practical  part  a  system  can  be  implemented  on  the  fly  without  the  crucial  waiting  for 
silicon.  We  will  not  go  into  more  detail  here  because  courses  at  this  level  are  well  known. 
This  part  of  the  lecture  is  completely  based  on  the  new  means  we  are  describing  below. 

Web  based  teaching 

As  mentioned  above,  switching  over  to  a  web  based  environment  introduces  a  lot  of  new 
possibilities  which  have  not  been  available  before.  In  the  following  we  will  show  the 
various  features  used  in  the  preparation  of  the  course  and  their  applications  and 
implications. 

The  first  idea  when  you  start  working  with  the  web  is  to  transfer  your  slides  which  you 
use  for  the  lecture  into  HTML  pages.  The  result  is  that  you  can  your  computer  and  an 
overhead  display  and  just  show  the  pages  without  producing  transparencies  using  a 
standard  web  browser.  It  might  give  a  good  feeling  but  the  information  really  needs  not  to 
reside  on  the  machine  you  are  using  for  the  presentation.  Further  on,  the  amount  of 
material  available  is  usually  much  more  than  what  you  can  present  in  one  course.  So  you 
can  select  exactly  that  information  which  is  relevant  for  you  for  your  lecture,  but  still  the 
students  can  if  they  wish,  go  through  all  available  information. 

The  more  interesting  aspect  is  the  fact  that  as  soon  as  you  make  the  web  pages  accessible 
to  the  students  they  have  the  opportunity  to  go  through  the  pages  in  advance  to  prepare 
for  the  lecture  and  can  trace  back  what  has  happend  after  the  lecture.  This  is  a  real  change 
in  the  style  of  teaching.  Following  the  conventional  approach  the  teacher  is  the  owner  of 
the  slides  and  the  students  have  to  take  a  textbook  for  further  reading.  This  strict  flow  of 
information  does  no  longer  exist  if  an  linked  stmcture  with  slides,  explanatory  text  and 
tutorial  parts  is  used.  The  resulting  structure  motivates  to  browse  through  the  information 
and  play  around  with  the  examples,  thus  making  it  much  easier  to  learn  than  in  a  strictly 
ordered  course.  Playing  is  much  more  effective  and  by  far  more  fun. 


Figure  1:  Example  of  a  lecture  transparency 


After  this  initial  step  the  contents  can  be  expanded  by  written  text.  The  series  of  sequen¬ 
tial  lecture  transparencies  can  serve  as  a  guide  through  the  material,  accompanied  by  text 
pages.  These  text  pages  must  be  associated  to  the  transparency  pages  and  switching  be¬ 
tween  the  two  information  types  must  be  made  easy.  A  structure  supporting  this  approach 
is  e.g.  a  ladder  as  illustrated  in  the  next  figure. 

The  entry  point  to  the  structure  is  the  first  lecture  transparency.  To  make  the  access  more 
convenient  links  from  the  directory  to  the  begin  of  the  chapters  and  subchapters  is  provi¬ 
ded.  These  links  are  omitted  in  the  sketch  below.  The  text  pages  themselves  lead  to  other 
problems  which  are  of  more  technical  kind:  The  size  and  the  resolution  of  the  screen.  The 
size  of  a  screen  is  usually  less  than  a  book  page.  To  avoid  that  the  user  has  to  scroll  up 
and  down  in  addition  to  moving  from  page  to  page  using  buttons  the  text  should  be 
limited  to  the  size  of  a  screen,  at  least  of  a  17"  monitor,  A  21"  Monitor  is  preferred, 
because  the  amount  of  text  then  gets  similar  to  that  what  can  be  printed  on  a  book  page. 

A  book  may  be  printed  on  a  laser  printer  of  600  dpi  or  better,  but  the  screen  usually  has 
only  72  dpi.  This  factor  of  (at  least)  10  limits  strictly  the  complexity  of  graphics.  One 
way  to  work  around  this  bottleneck  is  to  put  a  thumbnail  of  the  picture  in  the  text  and  put 
the  full  screen  drawing  on  a  separate  page.  This  methods  allows  to  roughly  figure  out  the 
contents  of  the  drawing  and,  if  something  is  not  clear,  to  look  into  the  detailed  version  of 
it. 


Figure  2:  Ladder  structure  of  transparencies  and  text  pages 

The  next  method  which  we  are  using  are  animations.  Usually  you  will  think  of  comic 
strip  figures  which  are  animated,  but  we  have  a  more  technical  view  here.  May  be,  in  the 
future...  In  the  technical  domain  there  a  many  flows  which  have  to  be  explained  to  the 
students.  In  all  these  cases  the  slide  in  the  lecture  can  only  show  a  snapshot  of  the  flow, 
but  never  really  the  dynamic  process  itself.  Usually  you  start  with:  At  the  initial  stage  ..., 
then:  This  step  is  shown  here...,  and  finally:  When  the  process  is  terminated,  then,  as  you 
can  imagine,... 

This  procedure  requiring  the  imagination  of  the  student  can  me  really  simplified.  In  the 
example  shown  below  we  have  taken  a  Petri  Net  describing  a  simple  b^ng  process, 
where  one  worker  produces  the  dough  whereas  the  other  one  prepares  the  rolls  and  puts 
them  in  the  oven.  At  the  beginning  both  dishes  are  empty.  Then  we  get  the  first  dough 
and  the  second  worker  can  start  his  job.  To  come  to  an  end,  the  first  worker  has  to  stop 
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reparing  dough  and,  after  all  dough  has  been  processed,  the  second  one  can  finish  work, 
too^. 

The  animation  is  written  as  a  Java  applet  so  that  every  browser  can  display  it.  The  flow  is 
controlled  by  the  two  buttons  next  and  reset,  ^^enever  you  click  the  next  button, 
you  go  to  the  next  state.  The  transitions  do  not  follow  directly  the  rules  of  the  Petri  Nets 
but  require  pressing  the  next  button  twice  to  get  to  the  next  state  of  the  net.  This  was 
introduced  to  give  ample  time  to  look  what  is  done  in  the  next  processing  step.  To  be  able 
to  control  the  flow  or  to  be  more  precise,  to  allow  the  two  guys  to  go  home  in  the  evening 
the  state  of  the  branch  points  can  be  changed  by  clicking  on  Ae  circles.  The  active  state  is 
indicated  by  a  short  thick  line  in  the  circle  which  shows  the  way  the  token  will  flow  when 
it  reaches  the  state  next.  Using  this  animation  the  teacher  can  illustrate  the  basic 
mechanism  of  a  Petri  Net  much  easier  and  the  student  can  play  with  the  animation  when 
he  is  on  his  own  machine  and  browsing  through  the  lecture  pages. 


Figure  3:  Petri  Net  animation 

Speedchart  Tutorial 

As  stated  before,  just  a  lecture  is  not  enough  to  teach  system  synthesis.  Working  with  the 
tools  is  absolutely  necessary  to  get  a  feeling  how  synthesis  is  performed.  So  there  is  the 
need  to  introduce  tools  into  the  course.  An  appropriate  tool  for  describing  systems  on  a 
high  level  is  Speedchart.  This  tool  allows  to  work  with  state  diagrams,  state  charts  and 
state  tables  to  describe  a  system.  For  those  parts  of  a  system  for  which  one  on  these 
approaches  are  not  appropriate  VDHL  or  Verilog  code  can  be  included,  too.  For 
speedchart  a  tutorial  has  been  prepared  so  that  the  students  can  work  on  their  own  after 
having  received  a  short  introduction.  The  following  figure  shows  a  bubble  diagram 
entered  using  speedchart. 


3  Entnommen  aus:  Siegfried  Wendt:  Informationstechnik 
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The  integration  of  tools  is  always  done  in  the  same  manner.  All  CAD  tools  used  in  this 
tutorial  have  an  X- Windows  GUI^  and  are  running  on  SUN  Workstation.  All  software  is 
protected  by  licensing  and  can  only  run  if  the  appropriate  licence  server  is  available.  A 
direct  interface  usable  by  the  web  browser  is  not  available.  The  way  how  we  solved  the 
problem  is  to  use  an  X-Client  on  the  local  machine.  The  complete  procedure  for  calling 
an  external  tool  is: 

0  •  Log  on  with  username  and  password.  This  is  required  because  the 
commercial  tools  can  be  used  for  teaching  but  must  of  course  not  be  made 
available  to  the  public. 

1  •  Start  your  X-Client 

2  •  Get  your  IP-Address  and  set  the  display  variable  of  the  host  machine 
running  the  CAD  software  to  your  BP- Address 

3  •  Activate  the  appropriate  link.  Doing  this  a  cgi  script  is  activated  which 
provides  the  correct  environment  for  the  CAD  tool  before  it  starts  the  tool 

After  having  performed  this  step  he  software  can  be  used.  The  experience  showed  that  if 
both  machines  are  workstations  the  feeling  is  as  if  you  were  directly  connected.  Other  X- 
Clients  are  significantly  slower,  even  with  a  high  speed  connection.  Using  a  slow  modem 
turned  out  to  be  a  little  boring  because  of  the  communication  overhead  of  the  X  protocol. 
Besides  this  it  works  nicely. 

VHDL  Online 

The  core  part  of  the  system  is  the  VHDL  Online  module  which  comprises  a  whole  set  of 
elements:  A  VHDL  tutorial,  an  annotated  reference  manual,  a  module  for  multiple  choice 
tests,  simple  examples  and  a  more  extended  example,  the  camera  control  unit. 


4  GUI:  Graphical  User  Interface 


441 


The  VHDL  tutorial  started  as  a  conventional  course.  The  materials  developed  for  the 
course  have  been  transferred  into  a  sequence  of  HTML  documents  and  form  the  basis  for 
this  part  of  the  complete  system.  The  amount  of  pages  is  still  growing,  because  not  only 
the  language  constructs  have  to  be  explained  semantically  but  the  examples  are  getting 
more  and  more  and  are  improving  in  quality.  When  the  student  analyses  the  examples  he 
will  mention  that  the  simulator  produces  error  messages.  This  is  intended,  and  after  a 
correction  according  to  the  error  message  he  will  be  able  to  analyse  it. 

To  help  the  students  when  they  are  not  sure  how  to  use  a  construct  a  reference  has  been 
built  up.  In  the  reference  the  constructs  are  listed  by  category  and  each  entry  has  an 
example  associated  to  it  describing  the  environment  where  it  is  used.  The  glossary 
contains  additional  information  around  VHDL  and  links  to  the  appropriate  keyword  if 
necessary. 

Finally,  a  workshop  is  provided  in  which  the  control  of  a  camera  is  designed.  Starting 
with  the  structure  the  modules  are  designed  one  by  one.  An  example  text  is  provided  for 
all  subtasks.  Within  that  workshop  all  concepts  of  VHDL  are  used,  stmctural  as  well  as 
behavioural  style  descriptions.  The  students  can  work  on  this  example  really  on  their 
own.  After  having  passed  the  workshop  they  know  enough  to  work  on  a  bigger  VHDL 
model  which  is  usually  the  core  of  the  practical  part  of  the  course. 

For  the  implementation  of  their  big  example  the  students  can  rely  only  on  their  own  code, 
but  within  the  project  a  library  of  synthesizable  models  is  provided.  Elements  are  carry 
look  ahead  adders,  array  dividers,  fifos,  a  parity  generator  and  various  counters. 

FPGA  Online 

Teaching  all  this  theoretical  contents  including  synthesis  is  good  if  only  the  principals 
shall  be  shown.  For  getting  a  better  feeling  a  real  technology  has  to  be  targeted  at.  In  our 
case  we  have  chosen  Xilinx  FPGAs^.  There  are  several  reasons  for  this: 

0  •  The  n^GAs  give  a  lot  of  limitations  with  respect  of  size  and  the  number  of 
pins.  In  this  case  it  is  a  real  advantage,  because  the  examples  which  are  small 
enough  to  fit  into  the  course  usually  can  easily  be  implemented  in  any  other 
technology. 

1  •  FPGAs  are  really  in  industrial  use  for  fast  prototyping.  So  the  constraints 
are  not  artifical  but  reflect  the  reality. 

2  •  The  availability  of  FPGAs  on  a  demo  board  permits  to  immediately  check 
out  in  reality  if  the  design  is  working  or  not.  W^en  a  standard  cell  design  is 
made  the  manufacturing  cycle  usually  is  so  long  that  the  students  have 
finished  their  course  month  ago  when  the  silicon  is  arriving. 

3  •  There  are  libraries  and  interfaces  available  to  link  the  synthesis  system  and 
the  Xilinx  tools. 

The  FPGA  Online  consists  of  two  parts,  an  introduction  into  FPGAs  and  an  introduction 
to  the  Xilinx  tools.  The  Tools  are  called  in  the  same  way  as  described  above  and  the  next 
figure  shows  how  it  looks  like  when  you  call  the  Xilinx  tools  for  placement,  routing  and 
programming. 


5  FPGA:  Field  Progranunable  Gate  Arrays 
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Figure  1:  Example  of  a  lecture  transparency 


Communication 

With  the  internet  advanced  communication  means  come  nearly  for  free.  Within  die 
course  we  are  using  two  of  them;  the  first  one  is  the  well  known  e-mail  communication 
which  allows  to  send  information  to  the  adviser  and  back  without  being  online.  This 
works  fine  but  lacks  the  possibility  of  immediate  response.  So  we  have  introduced  a  chat 
channel  where  the  teachers  are  available  at  a  certain  time.  This  is  not  limited  to  one 
location.  Having  a  common  time  for  all  locations  where  this  system  is  in  use  gives  the 
opportunity  to  talk  to  the  most  competent  person  among  the  instructors,  thus  increasing 
the  quality  of  the  course.  The  chat  communication  is  implemented  as  a  Java  applet  and 
thus  can  be  used  using  any  internet  browser. 

Conclusions 

The  system  presented  here  has  been  proved  workable  by  using  it  in  lectures  at  the 
Universities  of  Bremen,  Darmstadt,  Erlangen,  Frankfurt  and  Siegen.  The  general 
experience  was  very  positive  and  everybody  who  worked  with  the  system  continue  using 
it.  In  particular  the  ease  of  tailoring  the  system  to  the  needs  of  a  specific  course  is  a  big 
plus.  The  solid  support  of  practical  training  for  various  tools  helps  to  reduce  the  effort  of 
maintaining  the  examples  and  offers  a  much  wider  variety  as  could  be  provided  in  a  pure 
local  version. 

Nevertheless  the  system  is  by  far  not  complete.  First,  there  are  still  gaps  which  have  to  be 
filled.  The  bigger  problem  still  is  the  maintenance,  because  there  are  tools  missing  which 
really  support  cooperative  authoring  and  which  help  to  maintain  all  the  links  between  the 
various  parts  of  the  system.  This  is  a  point  where  many  changes  can  be  expected  in  the 
near  future. 
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Abstract 

The  discussion  of  the  article’s  intention  is  followed  by  a  brief  summary  of  the  status  of 
distance  learning  at  the  FemUniversitat  Gesamthochschule  at  Hagen.  Against  this  back¬ 
ground  the  further  explanations  have  to  be  seen.  They  want  to  demonstrate  how  a  multi- 
media  course  for  students  of  an  engineering  science  can  be  developed  using  a  systemati¬ 
cal  procedure.  Regarding  the  various  evaluation  methods  in  detail,  this  article  tries  to  em¬ 
phasize  the  importance  of  this  item,  which  has  been  neglected  in  so  many  other  compara¬ 
ble  projects  so  far. 

Introduction 

The  systems  approach  is  a  well  proved  method  for  developing  systems.  Because  of  the 
fact,  that  the  systems  approach  can  be  modeled  on  a  high  level  of  abstraction,  it  is  gener¬ 
ally  useful  for  many  disciplines.  The  working  scope  of  designing  educational  systems 
does  not  make  an  exception  (Issing,  1995).  But  the  more  you  want  to  cut  down  wide 
fields  of  action  (e.g.  because  of  the  desire  of  creating  a  single  educational  film  or  cur¬ 
riculum  for  a  special  subject),  the  more  an  abstractionally  defined  systems  approach  has 
to  loose  its  heuristical  power.  Therefore  it  is  necessary  to  elaborate  the  abstract  model  for 
the  considered  domain.  This  elaboration  is  as  valuable  as  the  abstract  modeling  of  the 
systems  approach  for  the  wide  working  scope.  For  the  task  of  developing  a  multimedia 
course  for  engineers  a  possible  elaboration  will  be  described  in  this  article. 

Status  of  Distance  Learning 

At  the  University  of  Hagen  teaching  has  mainly  been  based  on  letters  of  studies,  which 
are  similar  to  books.  Because  electrical  engineers  have  to  get  some  practice,  there  are 
additionally  to  these  so  called  "Studienbriefe",  exercises,  laboratories  and  seminars.  Up  to 
now,  these  events  have  taken  place  at  Hagen.  Thus  we  can  summarize,  that  there  is  a 
combination  of  distance  and  presence  learning  at  the  university. Furthermore  it  is  possible 
to  study  videos  for  special  items  (e.g.  solar  energy),  which  may  partly  replace  excursions 
and  practical  demonstrations. To  evaluate  new  or  redesigned  courses  formative  evaluation 
is  realized  by  the  so  called  "Lehrtextkritik",  which  means  a  questionnaire,  that  is  not  spe¬ 
cific  to  a  single  course  or  discipline  like  engineering.  The  implementation  of  the  so  called 
'Virtual  university",  that  is  internet  and  PC  based,  offers  new  possibilities  in  using  multi- 
media  aspects.  Especially  ET-Online,  the  part  that  has  been  created  by  the  department  of 
electrical  engineering,  has  been  working  with  several  synchronous  and  asynchronous 
communication  services.  At  the  moment,  students  can  discuss  with  the  departments’  col¬ 
laborators  by  using  for  example  electronic  mail,  newsgroups  and  chat.  In  addition  to  these 
kinds  of  online  support,  students  partly  can  get  the  letters  of  studies  in  a  multimedia  ver¬ 
sion  on  a  CD-ROM. 
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Theoretical  approach  for  the  design  of  a  new  multimedia  course  for  engineers 
The  systems  approach  is  a  proved  scientific  method  to  stmcture  the  process  of  developing 
a  system.  A  system  consists  of  a  number  of  elements  interchanging  information,  energy 
and  matter.  The  systems  approach  is  often  associated  with  the  idea  of  a  succession  of 
single  steps  in  a  totally  linear  procedure.  But  in  reality  it  is  imperative  to  work  with  repe¬ 
titions,  branching  and  feedback.  Otherwise  one  would  not  be  able  to  achieve  a  solution  in 
the  sense  of  the  systems’  goals.  The  systems  approach  is  used  to  be  modeled  by  the  basic 
model  of  the  general  system  design  (GSD-model), 


Because  of  the  complexity  of  single  phases  hke  analysis,  planning,  development,  usage 
and  evaluation/revision  it  may  be  supposed,  that  the  GSD-model  can  only  be  understood 
as  a  heuristical  problem  solving  method. 

The  systems  approach  has  been  used  for  educational  purposes  since  the  end  of  the  50*^. 
There  have  been  hundreds  of  articles,  which  publish  ways  of  putting  the  GSD-model  into 
concrete  terms.  The  result  is  an  instmctional  design  model  (ISD-model),  Gustaffson 
(Gustaffson,  1991)  divided  these  ISD-models  into 

•  ISD-models  for  developing  systems  of  education 

•  ISD-models  for  lessons 

•  ISD-models  for  the  development  of  products  and  media 

by  using  the  level  of  action  criteria.  Nevertheless  the  intention  of  the  above  mentioned 
groups  of  ISD-models  is  quite  different,  they  all  concur  more  or  less  in  the  basic  steps 
analysis,  planning,  development,  usage,  and  evaluation/revision. 

Directed  by  a  certain  ISD-model,  the  system  designers’  task  is  to  derive  an  appropriate 
instruction  design.  For  multimedia  courses,  the  last  two  groups  of  ISD-models  are  the 
most  important  ones. 

Elaboration  of  the  GSD-model  for  designing  a  new  multimedia  course  for  engineers 
It  was  mentioned,  that  teaching  is  partly  based  on  written  letters  of  studies,  which  are 
dispatched  each  term.  A  few  years  ago  some  departments  started  to  work  out  multimedia 
versions  of  their  courses.  In  this  paragraph  we  want  to  discuss  the  reflections,  that  are 
necessary  for  deriving  an  ISD-model  for  developing  such  a  multimedia  course  for  engi¬ 
neers.  For  this  purpose  we  subdivide  the  phases  of  the  GSD-model. 

The  exceptional  position  of  evaluation 

The  term  evaluation  is  defined  in  a  lot  of  different  ways,  because  evaluation  can  be  con¬ 
nected  with  many  totally  different  domains.  In  the  context  of  multimedia  courses, 
evaluation  means  first  of  all  regarding  the  efficiency  of  the  developing  process.  Addition¬ 
ally  it  deals  with  the  criteria  acceptance  and  effectiveness  of  multimedia  courses.  But 
independent  of  the  chosen  focus,  evaluation  is  always  more  than  only  collecting  data. 


Figure  1:  Basic  model  of  the  General  System  Design  (GSD-model) 
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Will,  Winteler  and  Knapp  (Will,  1986)  take  this  into  consideration,  when  they  talk  about 
the  four  main  aspects  of  evaluation: 

•  Evaluation  has  got  a  purpose  or  an  aim, 

•  Evaluation  is  based  upon  a  systematically  captured  data  base. 

•  Evaluation  always  includes  kind  of  a  judgement  that  is  result  of  using  statisti¬ 
cal  methods  on  the  data. 

•  Evaluation  is  not  interested  in  the  performance  of  a  single  subject  (e.g.  a  stu¬ 
dent),  but  it  means  aggregation  to  judge  about  a  special  item  like  a  multimedia 
course. 

Regarding  these  four  main  aspects  we  can  differentiate  several  kinds  of  evaluation.  The 
most  important  and  best  known  ones  are: 

•  Formative  Evaluation 

In  an  early  stage  of  the  developing  process  mistakes  and  deficiencies  can  be 
remedied  with  less  expense.  Therefore  it  is  advisable  to  involve  members  of 
the  user  group  in  the  developing  process.  The  observing  and  interrogating  of 
these  persons  helps  to  optimize  the  product  or  service.  The  risk  of  a  total  fail¬ 
ure  is  minimized  (Issing,  1995).  Alternative  to  formative  evaluation  one  can 
use  the  method  of  rapid  prototyping,  too  (Tripp,  1990).  The  method  imple¬ 
ments  system  functions  only  (most  of  the  time  software  functions)  in  a  way, 
that  is  sufficient  to  optimize  the  interface  between  user  and  system.  It  is  not 
necessary  to  implement  the  total  functionality  (Kraiss,  1986). 

•  Summative  Evaluation 

This  kind  of  evaluation  is  focused  on  the  completed  product  or  service.  It  is 
always  placed  at  the  end  of  the  developing  process.  If  we  remind  the  GSD- 
model,  we  are  talking  about  the  phase  of  usage.  In  comparison  with  formative 
evaluation  this  one  is  quite  more  expensive  and  has  a  higher  expenditure  of 
time. 

The  numerous  kinds  of  evaluation  are  listed  in  the  article  of  Gotz  (Gdtz,  1993),  At  this 
point  we  can  summarize,  that  evaluation  is  not  only  a  necessity  in  a  single  phase  (sum¬ 
mative  evaluation  in  the  usage  phase),  but  it  should  accompany  the  whole  developing 
process,  modeled  by  the  GSD-model. 

Analysis 

Learning  material  always  pursues  an  object.  These  aims  are  exposed  by  answering  the 
question,  what  should  be  changed  in  the  students’  way  of  thinking,  their  knowledge,  their 
skills  and  their  mental  attitudes  by  working  with  the  multimedia  course.  Without  precise 
definition  of  these  teaching  goals,  you  are  not  be  able  to  find  out,  whether  they  have  been 
attained  or  not. 

Such  a  specification  is  not  possible  without  having  a  closer  look  at  the  students'  attrib¬ 
utes.  It  is  impossible  to  define  any  aim,  especially  an  educational  one,  without  reflecting 
their  status  momentarily.  First  of  all,  it  is  quite  important  to  elaborate  the  decisive  signs. 
Then  you  should  be  informed  relative  to  what  sign  the  group  of  students  is  homogeneous 
or  heterogeneous. 
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For  example,  electrical  engineers  as  well  as  students  of  computer  science  are  more  fa¬ 
miliar  with  computers  than  other  groups,  because  it  is  part  of  their  courses  and  normally 
they  have  already  some  experiences  before  they  start  studying  such  disciplines.  This  ef¬ 
fect  can  be  used  with  some  benefit  for  the  introduction  of  a  multimedia  course. 
Furthermore  it  is  obvious,  that  the  goals  for  high  motivated  students,  who  have  already 
got  specialized  knowledge,  have  to  be  different  from  those  for  students  forced  to  partici¬ 
pate  because  of  the  curriculum. 

Additionally,  the  usage  of  multimedia  technologies  depends  on  the  students’  technical 
equipment.  For  example,  it  does  not  make  much  sense  to  produce  expensive  videos,  if  a 
high  percentage  of  students’  PCs  do  not  allow  to  play  them,  does  it?  But  nevertheless,  it  is 
recommended  to  estimate  future  evolutions. 

In  consequence,  a  personal  data  base  should  deliver  information  about  demographical, 
socio-economical,  psychographical  items  as  well  as  about  technical  equipment.  To  cap¬ 
ture  these  data  it  is  common  to  interview  the  students.  You  have  to  weigh  the  selected 
number  of  attributes  (which  of  course  correspond  with  the  size  of  your  questionnaire) 
against  the  quality  of  your  data. 

At  the  institute  of  automation  (LG  AT)  we  are  working  with  a  self-designed  question¬ 
naire. 

Compared  with  the  above  mentioned  ''Lehrtextkritik",  this  questionnaire  is  specifically 
designed  for  electrical  engineers  and  for  those,  who  are  interested  in  automation  technol¬ 
ogy.  An  extract  is  shown  below: 

•  Why  do  you  participate  in  this  course? 

O I  have  to  proof  a  certificate. 

O I  have  got  a  general  interest  in  the  subject. 

O I  have  to  qualify  myself  for  my  job. 

O I  will  start  a  new  job  dealing  with  this  subject. 

O... 

•  How  would  you  describe  your  specialized  knowledge  at  the  moment? 

Excellent  O  O  O  O  O  O  Bad 

•  Do  you  cooperate  with  your  fellow  students? 

OYes 

ONo 

•  What  kind  of  technical  aids  do  you  use  for  cooperation  with  your  fellow  stu¬ 
dents? 

O  Telephone 
OFax 
O  Email 
O.... 
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•  What  do  you  prefer  working  with  your  fellow  students? 

O  Discussing  the  content  of  the  letter  of  studies. 

O  Doing  exercises 

O  Preparing  myself  for  the  exame 

O.... 

•  Would  you  like  to  get  any  support  using  the  following  services? 

O  Email 

OFTP 
O  Telnet 
OChat 
OWWW 
O... 

Having  a  closer  look  at  these  examples  out  of  our  questionnaire,  you  can  realize,  that  the 
measuring  level  can  be  quite  different.  That  means,  you  have  to  be  very  careful  using 
statistical  methods.  You  can  dismiss  the  result  of  a  statistical  approach,  if  the  statistical 
method  does  not  suit  the  measuring  level. 

So  far,  we  have  only  discussed  appropriate  ways  of  analyzing  the  students’  attributes.  But 
for  the  phase  of  analysis  of  the  GSD-model  some  additional  work  has  to  be  done.  Devel¬ 
oping  a  multimedia  course  for  electrical  engineers,  it  is  very  helpful  to  have  a  closer  look 
at  the  special  requests  of  the  occupational  life  of  electrical  engineers.  This  part  of  analysis 
we  based  on  a  working  sheet,  like  the  one  you  can  see  in  figure  2. 


Requirements  of  occupational  life 

Discipline 

Natural 

science 

Engineering 

Economics 

... 

Ability  to  get  familiar  with  new  domains 

/ 

Ability  to  abstract 

*■«* 

A** 

Ability  to  communicate 

2 

»*• 

3-D  imagination 

*•* 

legend: 

1  ^  very  important 

2  =  important 

3  =  ... 

Figure  2:  Working  sheet  to  eleborate  the  importance  of  special  requests 


It  has  to  be  said  ,  that  the  working  sheet  in  figure  2  is  a  general  one,  because  it  offers  to 
regard  other  subjects  than  engineering  as  well. 

In  this  paragraph  it  is  shown  that  for  a  multimedia  course  teaching  goals  shall  be  defined 
against  the  background  of  reflections  to  students’  attributes,  technical  equipment  and  spe¬ 
cial  requests  of  the  occupational  live. 
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Planning 

The  phase  of  planning  from  the  GSD-model  focuses  on  achievement  of  the  defined  goals. 
In  this  context  we  can  list  four  important  factors  (Minnesota  Research  Center,  1990): 

•  Fidelity 

Means  presenting  contents  in  a  realistic  and  problem  driven  way 

•  Visualization 

Means  presenting  contents  close  to  the  students’  world 

•  Range  and  depth  of  experience 

Means  presenting  contents  out  of  different  points  of  view, 

•  Mediation 

Means  emphasizing  essential  contents 

The  mentioned  factors’  character  show,  that  you  have  to  consider  thoroughly  the  courses’ 
contents.  This  consideration  has  to  define  the  relevant  parts  of  the  courses  contents, 
which  are  to  be  trickily  connected  with  didactic  elements  in  the  factors’  sense.  But  this  is 
a  quite  difficult  task,  because  there  is  a  great  variety  of  these  elements  using  multimedia 
(e.g.  in  addition  to  common  didactic  elements  you  have  got  videos,  simulation,  sound  and 
so  on)  and  there  exists  no  detailed  prescription  how  to  do  this. 

Therefore,  first  of  all  we  think  about  appropriate  measures,  that  will  qualify  the  students 
to  meet  the  requirements.  Here,  we  use  another  worksheet,  shown  in  figure  3. 


Working  life  requirements 

Qualification  measures 

Natural 

science 

Engineering 

Economics 

... 

Ability  to  familiar  with  new  domains 

B 

Ability  to  abstract 

Exercises  for 
training  niodet>* 
ling 

4i9M 

Ability  to  communicate 

lililiilii 

K»M 

3-D  imagination 

A¥* 

Figure  3:  Working  sheet  to  elaborate  qualification  measures 


Then  in  a  second  step,  we  combine  these  founded  qualification  measures  with  the  didac¬ 
tic  elements  of  multimedia. 

To  illustrate  the  above  mentioned,  furthermore  will  be  pointed  out  a  global  teaching  goal 
called  "Fachkompetenz".  If  students  are  able  to  use  their  work  equipment  properly  and 
can  control  working  processes  based  on  their  specialized  knowledge,  they  will  reach  this 
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goal. 


An  elaboration  of  this  goal  naming  suitable  indicators  is  given  remembering  the  contents 
of  the  working  sheet  in  figure  2.  Regarding  the  there  mentioned  indicator  "Ability  to  ab¬ 
stract",  one  appropriate  qualification  measure  would  be  a  couple  of  exercises,  which  train 
modeling  (figure  3). 

Which  kind  of  didactic  element  trains  students  in  modeling?  A  possible  answer  to  this 
question  is  the  integration  of  simulation  environments  in  a  multimedia  course  for  engi¬ 
neers. 

A  simulation  is  a  special  form  of  an  interactive  program.  It  is  based  on 

•  a  model  of  the  viewed  system  and 

•  a  number  of  equations  that  determine  the  dynamic  features  of  the  system. 

There  are  several  ways  to  elaborate  the  simulation  environment,  starting  with  displaying 
only  a  static  model  and  the  equations,  up  to  graphical  displaying  of  the  dynamic  behavior 
including  a  modification  of  the  parameters  by  die  students  (remember  the  factors  "Fidel¬ 
ity"  and  "Visualization").  Because  of  the  fact,  that  the  cognitive  requirements  of  such 
environments  can  be  very  high  it  is  recommended  to  think  about  a  certain  degree  of  in¬ 
structional  support. 

Depending  on  the  specific  elaboration  the  students  will  have  more  or  less  opportunities 
for  an  active  integration  of  the  theoretical  contents  in  their  so  far  existing  mental  models. 
When  we  follow  the  didactic  theory  of  constructivism,  theoretical  contents  have  to  be 
experienced  and  can  not  only  be  put  over  to  the  students’  minds. 

Finally,  you  can  see  an  example  of  a  simulation  environment  in  figure  4. 


Figure  4:  Simulation  environment  for  the  new  multimedia  course 


Human-machine  interaction  is  an  important  part  of  automation  technology  today.  There¬ 
fore  it  is  necessary  to  learn,  how  human-machine  interfaces  are  designed  and  integrated 
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in  the  automation  system.  Using  a  well  known  HMI-tool,  the  student  has  to  design  a  HMI 
for  a  specific  example.  The  shown  panel  consists  of  the  typical  elements,  e.g.  function 
keys  (bottom)  to  switch  from  one  panel  to  another,  a  scheme  of  the  plant  (background  , 
which  has  to  be  designed  in  a  graphic  editor),  and  the  dynamic  values  shown  as  bar 
graphs  (wet  ingredients),  and  the  status  of  a  mixer  (indicated  by  color)  as  well  as  alarms 
(upper  part)  and  acknowledgement.  Additionally,  the  produced  batch  "Chocolate  Chip"  is 
shown. 

We  will  integrate  such  an  simulation  environment  into  our  new  multimedia  course  for 
electrical  engineers. 

Development 

In  the  developing  phase  of  the  GSD-model,  the  systems  approach  gets  into  a  more  opera¬ 
tive  character.  The  teaching  goals  (analysis  phase)  combined  with  the  theoretical  reflec¬ 
tions  how  to  achieve  them  (planning  phase)  have  to  be  integrated  in  a  concrete  instruction 
design.  Considering  the  intention  of  working  out  a  multimedia  course  this  means  to  de¬ 
termine  the  look  of  the  user  interface. 

For  this  purpose  we  will  use  the  method  of  rapid  prototyping  which  was  already  men¬ 
tioned  in  the  paragraph  about  evaluation  work.  We  decided  to  use  the  tool  MS- 
Powerpoint©,  which  allows  not  only  to  create  panels  but  also  to  simulate  the  dialogue 
between  the  user  and  the  system.  Unfortunately,  there  are  again  only  heuristic  hinds  of 
disciplines  like  software-ergonomics  (e.g.  DIN  66234)  and  psychology,  so  that  an  itera¬ 
tive  design  process  is  unavoidable;  which  means  having  repetitions,  branching  and  feed¬ 
back. 

A  part  of  the  new  interface  design  out  of  an  early  step  of  iteration  is  displayed  in  figure  5. 
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Each  version  of  the  user  interface  will  be  formative  evaluated  using  the  following  meth¬ 
ods  of  test: 

•  Inspection  by  a  team  of  experts 

A  team  consisting  of  didactic  experts,  screen  designer  and  subject  experts 
evaluate  the  user  interface  considering  transparency  and  consistency.  Addi¬ 
tionally  it  is  important  to  know  whether  it  is  self-explaining. 

•  Observing  the  behavior  of  students’  representatives 

Only  students’  representatives  can  give  a  feedback  about  the  acceptance  of  the 
user  interface.  Tlierefore  student  co-workers  of  the  institute  of  automation 
will  participate  in  small  test  designs  where  they  will  have  to  solve  small 
problems.  During  the  test  their  navigation  activities  will  be  recorded.  Finally 
in  an  open  interview  they  can  give  impressions,  criticism  and  recommenda¬ 
tions. 

The  so  optimized  instruction  design  will  be  written  down  in  a  scenario.  This  scenario  is 
thought  to  be  handed  over  to  the  producer  of  the  new  multimedia  course.  Therefore  it  has 
to  specify  the  following  aspects  in  detail: 

•  Structure  of  dialogue 

The  structure  of  dialogue  can  be  reported  by  determining  a  network  of  navi¬ 
gation  possibilities  between  the  user  interface  elements. 

•  User  interface 

Each  panel  of  the  user  interface  has  to  be  described  including  the  use  of  but¬ 
tons,  frames,  pull  downs  and  so  on 

•  Media  production 

Listing  of  the  desired  production  of  videos,  photos,  sound,  computer  graphics 
and  simulations 

At  this  point  we  do  not  conceal,  that  the  production  of  these  media  has  a  strong  influence 
on  the  expenses  for  a  multimedia  course  on  a  high  degree.  On  the  other  hand,  even  media 
like  video  sequences  and  simulation  support  the  integration  into  the  students’  mental 
models  (Kozma,  1991). 

In  additon  to  our  example  simulation  environment  think  of  a  video  sequence  showing 
plant  facilities,  including  the  integration  of  elements  like  PLC  (Programmable  Logic 
Controller),  actuators  and  sensors. 

Usage  including  summative  evaluation 

After  finishing  the  new  multimedia  course  for  electrical  engineers  it  wiU  be  provided  at 
the  beginning  of  the  half  year  term  WS  98/99.  At  the  end  of  the  term  the  students  will  be 
requested  to  answer  a  questionnaire.  We  think,  that  it  will  be  better  to  start  evaluation 
work  at  a  time,  students  will  have  got  familiar  with  the  new  course.  Otherwise  the 
evaluation  data  are  influenced  on  a  high  degree  on  emotional  or  start-up  aspects.  This 
questionnaire  will  deal  with  the  following  domains: 
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•  Expectations  of  the  students 

What  do  you  expect  from  a  multimedia  course? 

•  Acceptance  of  the  course 

Regarding  your  expectations,  how  do  you  judge  this  multimedia  course? 

•  Success 

Did  you  think  it  is  more  efficient  to  work  with  this  new  multimedia  course 
compared  with  the  paper  course? 

When  we  decided  to  work  with  a  questionnaire  including  open  questions  we  kept  in 
mind,  what  Schulmeister  says  about  this  method  of  evaluation.  He  thinks,  that  it  is  very 
critical  to  base  an  evaluation  mainly  on  students’  statements  because  of  their  insufficient 
to  express  themselves  (Schulmeister,  1995).  But  with  a  questionnaire,  combining  both 
kinds  of  questions  and  the  possibility  to  inquiry,  we  are  convinced  to  get  a  usable  data 
base. 

Conclusions 

This  article  demonstrates  the  advantages  of  the  general  systems  approach,  modeled  by  the 
GSD-model,  developing  a  new  multimedia  course  for  electrical  engineers.  Following  the 
phases  of  analysis,  planning,  development,  usage  and  evaluation/revision  we  developed 
an  instmctional  design  system  model  (ISD-model).  Our  ISD-model  fits  very  well  in  the 
context  of  developing  multimedia  courses  for  electrical  engineers.  Partly  we  pointed  out, 
how  to  use  this  ISD-model  by  the  aid  of  working  sheets.  The  further  elaboration  of  the 
phases  by  specifying  tasks  and  working  sheets  will  be  part  of  our  future  work.  Finally  we 
do  not  want  to  conceal,  that  our  ISD-model  can  not  be  more  than  a  heuristic  aid,  although 
we  will  put  it  in  concrete  terms  on  a  high  degree.  Following  Issing,  who  talks  about  the 
science  of  learning  and  the  art  of  teaching  (Issing,  1995)  we  are  convinced,  that  the  in¬ 
struction  design  developing  process  cannot  be  totally  automated. 
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Abstract 

Binocular  vision  requires  that  the  vergence  angle  (p  between  the  visual  axes  of  the  two 
eyes  (Figure  la)  is  adjusted  by  the  extraocular  muscles  in  order  to  fuse  the  two  retinal 
images.  Even  when  fixating  a  target  binocularly,  errors  in  vergence  angle  (referred  to  as 
fixation  disparity)  may  occur  in  individuals  and  can  be  associated  with  visual  strain.  We 
extended  clinical  optometric  procedures  by  measuring  fixation  disparity  with  dichoptically 
presented  nonius  lines  at  viewing  distances  of  20  to  100  cm,  so  that  both  vergence  and 
accommodation  were  stimulated  adequately  as  in  normal  vision  at  the  workplace.  As  the 
viewing  distance  was  shortened,  mean  fixation  disparity  in  34  subjects  changed  monotoni- 
cally  from  1  minute  of  arc  eso  (i.e.,  the  eyes  converged  in  front  of  the  target)  to  -  3  min 
arc  exo  (i.e.  convergence  behind  the  target).  Fixation  disparity  was  also  measured  at  a 
constant  viewing  distance  of  40  cm,  while  varying  the  power  of  prisms  in  front  of  the  eyes 
to  change  the  vergence  angle.  Fixation  disparity  was  more  strongerly  affected  by  prisms 
than  by  viewing  distances  that  induced  the  same  vergence  angles  (which  can  be  explained 
by  accommodative  vergence),  however  the  slopes  of  these  two  types  of  fixation  disparity 
curves  were  correlated. 

The  change  of  fixation  disparity  with  distance  differed  reliably  among  subjects  with  normal 
binocular  vision.  Those  subjects  with  a  large  exo  disparity  at  near  fixation  adopted  longer 
viewing  distances  in  the  course  of  a  laboratory  near-vision  task.  These  results  suggest  that 
near  vision  problems  can  be  caused  by  a  fixation  disparity  at  near  distances;  in  such  cases, 
visual  displays  should  be  used  at  longer  viewing  distances. 

Introduction 

Observing  visual  displays  at  man-machine  interfaces  is  in  no  way  a  process  of  passive  in¬ 
take  of  information.  Rather,  the  eye  muscles  continuously  have  to  adjust  the  eyes  so  that 
the  point  of  regard  is  projected  onto  the  fovea  of  each  eye,  i.e.  the  center  region  of  the 
retina  with  the  best  spatial  resolution.  The  present  paper  deals  with  the  vergence  function 
of  the  eyes:  the  horizontal  external  eye  muscles  have  to  turn  the  visual  axes  relative  to 
each  other,  depending  on  the  actual  viewing  distance  (Figure  la).  If  we  look  at  infinity  our 
visual  axes  are  parallel.  The  resting  position  of  vergence  varies  among  individuals  with  an 
average  of  about  3.5  deg,  corresponding  to  a  distance  of  1  m.  The  closer  the  target  the 
stronger  the  force  that  the  extraocular  muscles  have  to  exert  in  order  to  turn  the  eyeballs 
towards  each  other.  If  you  move  this  piece  of  text  continuously  closer  to  your  eyes,  you 
immediately  feel  the  increasing  effort  that  is  needed  until,  at  a  certain  point,  double  vision 
occurs,  thus  the  ocular  muscles  are  no  longer  able  to  superimpose  the  images  of  the  two 
eyes. 
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The  aim  of  the  present  study  was  to  measure  the  effect  of  viewing  distance  on  the  ver- 
gence  function  of  the  two  eyes  and  to  investigate  implications  for  the  choice  of  a  comfort¬ 
able  viewing  distance  for  working  at  visual  displays. 

•  In  a  first  experiment,  it  was  shown  that  the  viewing  distance  has  an  effect  on  how 
precisely  the  external  eye  muscles  are  able  to  converge  the  two  visual  axes  to  the  actual 
fixation  point.  This  error  in  vergence  angle  (i.e.  fixation  disparity)  as  a  function  of  viewing 
distance  is  compared  with  similar  measurements  as  a  function  of  prisms  in  front  of  the 
eyes,  as  used  in  conventional  optometric  test  procedures. 

•  A  second  experiment  investigated  whether  fixation  disparity  as  a  function  of  view¬ 
ing  distance  can  predict  the  viewing  distance  that  subjects  find  comfortable  during  a  labo¬ 
ratory  near-vision  task. 

Fixation  disparity  as  a  function  of  viewing  distance  and  prism  load 
As  described  before  and  illustrated  in  Figure  la,  the  viewing  distance  d  and  the  resulting 
vergence  angle  (p  determine  the  load  (or  stress)  on  the  vergence  eye  muscles,  in  a  similar 
way  to,  e.g.  a  weight  lifted  by  the  arms  in  the  case  of  skeletal  muscles.  However,  in  clinical 
optometry  (e.g.  Sheedy  and  Saladin,  1993),  a  different  concept  of  testing  vergence  under 
load  is  used  which  is  illustrated  in  Figure  lb.  Instead  of  changing  the  viewing  distance,  one 
uses  a  fixed  viewing  distance  do  and  varies  the  power  and  orientation  of  prisms  that  are 
placed  in  front  of  the  eyes.  A  comparison  of  Figure  1  a  and  b  shows  that  the  same  ver¬ 
gence  angle  (pj  is  adjusted  by  the  eyes  by  either  having  a  short  viewing  distance  d]  or  by 
having  an  intermediate  viewing  distance  do,  but  with  an  appropriate  prism  in  front  of  the 
eyes  which  -  in  case  of  the  prism  bases  oriented  outward,  i.e.  "base- out”  -  turn  the  visual 
axes  inward  and  increase  the  vergence  angle  from  (po  to  (pj.  Reversed  prisms  ("base-in") 
induce  a  smaller  vergence  angle. 


Figure  1:  Vergence  angles  induced  by  viewing  distances  (a)  and  prisms  in  front  of  the 
eyes  (b). 
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This  prism  technique  of  testing  the  vergence  system  under  load  is  known  in  optometry  as 
"forced  vergence"  (e.g.,  Sheedy  and  Saladin,  1983).  A  possible  drawback  of  this  proce¬ 
dure  is  that  the  viewing  distance  is  kept  constant,  thus  the  stimulus  for  accommodation 
(the  focus  mechanism  of  the  eye)  is  also  constant.  This  differs  from  the  natural  way  of 
seeing  where  a  change  in  vergence  always  coincides  with  a  change  in  accommodation. 
Therefore,  it  was  the  aim  of  the  first  experiment  to  compare  how  the  vergence  system  re¬ 
sponds  when  the  load  is  varied  by  changing  the  viewing  distance  (as  shown  in  Figure  la), 
or  by  changing  the  prism  power  (and  orientation)  at  a  constant  viewing  distance  (as  in 
Figure  lb).  Table  1  describes  the  viewing  distances  and  prisms  that  where  chosen  to  have 
the  same  vergence  angles  in  the  two  experimental  conditions  (for  a  subject  with  an  inter¬ 
pupillary  distance  of  60  mm). 


Vergence  angle  (p 

Prism  power  at  40  cm 

17.1  deg 

15  prism  dioptre  (base-out) 

11.4  deg 

5  prism  dioptre  (base-out) 

40  cm 

8.6  deg 

0 

60  cm 

5.7  deg 

-  5  prism  dioptre  (base-in) 

100  cm 

3.4  deg 

-  9  prism  dioptre  (base-in) 

Table  1:  Viewing  distances  and  corresponding  prism  powers  used  in  the  experiment. 

The  dependent  variable  that  was  measured  to  describe  the  vergence  fimction  at  different 
viewing  distances  and  prism  powers  was  the  fixation  disparity,  i.e.  the  error  in  vergence 
angle  that  may  occur  when  a  fusion  stimulus  is  fixated.  Only  in  subjects  with  optimal  bin¬ 
ocular  vision  is  the  fixated  target  imaged  onto  the  center  of  the  fovea  in  each  eye,  so  that 
the  principle  visual  directions  of  both  eyes  intersect  at  the  fixation  point  (Figure  2,  left). 
Many  subjects  with  normal  binocular  vision  (indicated  by  good  stereoscopic  acuity)  may 
have  slight  deviations  from  this  optimal  state.  These  small  errors  in  vergence  typically 
amount  to  a  few  minutes  of  arc;  they  are  smaller  than  the  Panum  areas  and,  therefore,  do 
not  lead  to  double  vision.  Fixation  disparity  is  called  "eso"  or  "exo"  depending  on  whether 
the  eyes  converge  slightly  in  front  of  or  behind  the  fixation  point,  respectively,  as  illus¬ 
trated  in  Figure  2.  Subjects  with  large  fixation  disparities  tend  to  have  more  eyestrain  and 
asthenopic  complaints,  but  this  need  not  be  true  in  all  individuals  (Sheedy  and  Saladin, 
1983;  Pickwell,  1989). 


Figure  2:  Possible  conditions  of  fixation  disparity  6;  exo  is  defined  as  negative  (see  text). 
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Figwe  3:  Method  for  measuring  fixation  disparity  [from  Bonacker  et  al  (1994)]. 

As  shown  in  Figures  2  and  3  we  measure  fixation  disparity  psychophysically  by  presenting 
both  the  fusion  stimulus  (0X0,  visible  for  both  eyes)  and  two  dichoptic  nonius  lines 
(made  visible  separately  one  to  each  eye  by  means  of  polarizing  filters)  on  a  monochrome 
cathode  ray  tube  of  a  personal  computer  (Bonacker  et  al.,  1994).  If  a  fixation  disparity  is 
present  the  nonius  lines  must  have  a  physically  horizontal  offset  in  order  to  coincide  with 
the  principle  visual  directions,  to  be  imaged  onto  the  center  of  the  fovea  and  to  be  per¬ 
ceived  in  line.  This  physical  offset  between  the  nonius  lines  is  determined  with  an  adaptive 
psychometric  procedure  by  presenting  the  nonius  lines  in  a  series  of  short  tests  of  100  ms 
duration,  varying  the  amount  of  horizontal  offset  and  having  the  subjects  say  whether  the 
upper  line  is  perceived  to  the  right  or  to  the  left  of  the  lower  line. 

Measurements  in  a  group  of  34  young  adult  subjects  with  good  monocular  and  binocular 
vision  revealed  the  following  results  (Jaschinski,  1997).  As  the  viewing  distance  was 
shortened  from  100  to  20  cm,  mean  fixation  disparity  changed  from  1  min  arc  eso  to  -  3 
min  arc  exo.  Figure  4  shows  three  individual  examples  of  fixation  disparity  curves  de¬ 
scribing  the  extent  of  change  in  fixation  disparity  with  vergence  angle,  approximated  by 
regression  lines  separately  for  the  distance  and  the  prism  condition.  Both  the  prism  curve 
and  the  distance  curve  were  rather  flat  in  Subject  JK  and  much  steeper  in  Subject  KM; 
Subject  MU  is  an  intermediate  case.  The  slope  of  the  prism  curves  and  the  distance  curves 
were  correlated  across  subjects  (r==0.39;  p=0.024;  n=25).  However,  the  prism  curves  were 
significantly  steeper  than  the  distance  curves  (t=6.63;  p<0.001).  Thus,  using  base-in  and 
base-out  prisms  induced  larger  amounts  of  eso  and  exo  fixation  disparity,  respectively, 
than  when  the  same  vergence  angles  were  induced  by  viewing  distances.  This  can  be  ex¬ 
plained  by  the  different  way  of  stimulating  accommodation. 
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Repeated  measurements  in  the  same  subjects  showed  that  the  slope  of  the  distance- 
dependent  fixation  disparity  curves  differed  reliably  among  individuals,  as  shown  by  a  sig¬ 
nificant  test-retest  correlation  of  r  =  0.65  (p  <  0.0001;  n  =  34).  Thus,  some  subjects  (e.g. 
JK  in  Figure  4)  had  a  small  fixation  disparity  irrespective  of  viewing  distance,  while  others 
showed  a  marked  shift  of  fixation  disparity  in  the  exo  direction,  if  the  viewing  distance  was 
shortened  (e.g.  KM). 


Figure  4:  Three  individual  cases  of  fixation  disparity  as  a  function  of  vergence  angle 
(Ci  reels:  varying  viewing  distance;  triangles:  varying  prism  power). 

The  prism  curves  in  Figure  4  represent  examples  of  what  is  termed  in  optomety  "forced- 
vergence  fixation  disparity  curves".  From  clinical  experience  and  research  (e  g.  Sheedy  and 
Saladin,  1983)  it  is  known  that  those  subjects  with  steeper  and  more  a  symmetrical  prism- 
dependent  fixation  disparity  curves  are  more  likely  to  suffer  from  eyestrain  during  visual 
work.  Since  the  slopes  of  the  prism  curves  and  the  distance  curves  were  correlated,  the 
following  experiment  was  designed  to  investigate  possible  ergonomic  implications  of  the 
distance-dependent  fixations  disparity  on  working  at  a  near  visual  display  unit  (Jaschinski, 
in  press). 

Fixation  disparity  and  the  preferred  viewing  distance  in  a  near-vision  task 
We  measured  fixation  disparity  at  viewing  distances  of  30,  40,  and  60  cm  in  a  group  of  33 
young  adult  subjects  with  normal  vision.  The  group  was  divided  in  3  subgroups  depending 
on  the  extent  to  which  fixation  disparity  changed  with  viewing  distance.  In  the  1 1  subjects 
of  Subgroup  A  (closed  symbols  in  Figure  5a)  the  exo  change  was  more  than  3  min  arc 
when  the  viewing  distance  was  shortened  from  60  to  30  cm;  in  the  1 1  subjects  of  Sub¬ 
group  B  this  change  was  less  than  1 . 1  min  arc  (open  symbols  in  Figure  5b).  The  remaining 
1 1  subjects  had  intermediate  values.  We  had  the  subjects  perform  a  task  of  60  minutes 
duration  at  a  visual  display  unit  and  let  them  start  with  an  initial  viewing  distance  of  about 
40  cm  (which  is  appropriate  to  the  chosen  character  height  of  2.4  mm).  But  during  the 
course  of  the  task  subjects  were  free  to  adopt  any  comfortable  viewing  distance,  which 
was  measured  continuously  with  a  video  system.  Figure  5c  shows  mean  values  of  viewing 
distance  for  each  5 -minutes  interval.  Most  of  the  subjects  in  Subgroup  A  (those  who  had  a 
larger  exo  fixation  disparity  at  shorter  viewing  distances)  moved  further  away  from  the 
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screen  and  adopted  longer  viewing  distances  compared  to  the  subjects  of  Subgroup  B, 
whose  fixation  disparity  remained  smaller  when  changing  the  viewing  distance.  An  analysis 
of  variance  with  repeated  measures  revealed  that  the  time  course  of  viewing  distance  was 
significantly  different  in  the  two  subgroups;  F(1 1,220)  =  2.94;  p  =  0.027  with  Greenhouse- 
Geisser  correction.  The  tendency  of  the  subjects  in  Subgroup  A  to  adopt  longer  viewing 
distances  is  interpreted  as  a  way  to  avoid  larger  fixation  disparities.  Most  users  with  large 
exo  fixation  disparity  at  near  did  not  accept  a  short  viewing  distances. 


CC 


30  40  50  60 

viewing  distance  (cm) 


0  30  60  min 

time  on  task 


Figure  5:  Fixation  disparity  as  a  fmiciion  of  viewing  distance  in  two  subgroups  (a  and  h) 
and  time  course  of  viewing  distance  during  a  near-vision  task  in  these  groups  (c). 

Conclusions 

Inappropriate  stress  on  the  vergence  system  of  the  eyes  must  be  avoided,  in  order  to  re¬ 
duce  visual  discomfort  and  fatigue  in  demanding  visual  tasks  at  human-machine  interfaces. 
While  clinical  optometric  methods  test  the  vergence  system  under  load  by  measuring  the 
fixation  disparity  as  a  function  of  prisms  in  front  of  the  eyes  and  keeping  the  viewing  dis¬ 
tance  constant,  the  present  research  introduces  the  measurement  of  fixation  disparity  as  a 
function  of  viewing  distance  which  is  directly  related  to  the  vergence  angle. 


The  rate  of  change  in  fixation  disparity  with  vergence  angle  was  analysed  by  calculating 
the  slope  of  fixation  disparity  as  a  function  of  prism  power  and  viewing  distance.  Both 
types  of  slopes  were  moderately  correlated;  however,  the  slopes  were  smaller  in  the  dis¬ 
tance  condition  as  compared  to  the  prism  condition,  which  can  be  explained  by  the  mecha¬ 
nism  of  accommodative  vergence.  Measuring  fixation  disparity  as  a  function  of  viewing 
distance  has  the  advantage  of  preserving  the  natural  interaction  between  accommodation 
and  vergence,  and  thus  better  reflects  the  viewing  conditions  in  everyday  vision  and  at  the 
workplace.  The  procedure  of  testing  the  vergence  system  as  a  function  of  prism  load  at  a 
constant  viewing  distance  resembles  the  condition  at  stereoscopic  displays  where  vergence 
is  changed  by  varying  the  disparity  of  stimuli  presented  separately  to  the  two  eyes  at  a 
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constant  viewing  distance.  Thus,  the  resulting  fixation  disparity  may  be  comparable  in 
these  two  conditions  (Duwaer,  1983). 

It  was  found  that  subjects  with  normal  binocular  vision  differ  reliably  in  the  rate  of  change 
in  fixation  disparity  with  viewing  distance,  i.e.  the  slope  of  the  distance-dependent  fixation 
disparity  curves.  In  a  laboratory  near-vision  task,  subjects  with  steeper  slopes,  thus  larger 
exo  fixation  disparity  at  near  tended  to  increase  the  viewing  distance  during  a  near  vision 
task  more  than  other  subjects  with  smaller  slopes.  This  is  interpreted  as  a  strategy  to  avoid 
a  condition  that  causes  a  large  fixation  disparity. 

This  results  can  be  compared  with  research  on  the  resting  position  of  vergence,  usually 
called  dark  vergence  since  it  is  measured  in  darkness  in  order  to  exclude  any  flisional  and 
accommodative  stimulus.  Heuer  et  al.  (1989)  reported  that  those  subjects  who  had  a  more 
distant  dark  vergence  adopted  longer  viewing  distances  while  working  at  a  visual  display, 
presumably  in  order  to  reduce  the  discrepancy  between  the  viewing  distance  and  their  dis¬ 
tant  dark  vergence.  Subjects  with  a  distant  dark  vergence  typically  have  a  larger  exo  fixa¬ 
tion  disparity  in  near-vision  (Jaschinski-Kruza,  1994). 

Thus,  the  exo  fixation  disparity  induced  by  near  visual  targets  and  dark  vergence  are  idio¬ 
syncratic  optometric  properties  that  may  have  implications  for  the  ergonomic  design  of 
workstations  with  near  visual  displays.  Individual  differences  in  the  vergence  system  could 
be  considered  when  adapting  the  workstation  design  to  the  individual  user. 
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Abstract 

Simulation  studies  are  successfully  used  as  assisting  tools  in  the  field  of  operation  sched¬ 
uling  and  work  planning.  Work  plans  generally  supply  the  most  important  input  data  for 
simulation  studies  in  this  field.  They  determine  the  combination  of  the  technical  compo¬ 
nents  of  a  work  system  and  the  order  of  manufacturing  steps  towards  a  product.  Simula¬ 
tion  studies  support  the  labor  organization  of  a  work  system  from  the  view  of  high  effec¬ 
tiveness  and  economic  efficiency  in  a  given  time  period.  Missing  information  about 
strains  and  risks  a  person,  which  is  the  worker,  might  be  exposed  to  leads  to  a  loss  of 
quality  of  the  used  models.  Health  risks  can  jeopardize  the  economy  of  production  sys¬ 
tems,  too,  if  the  person  is  considered  as  a  subsystem. 

On  the  one  hand,  the  simulation  of  operations  in  production  and  logistics  requires  human 
related  input  data.  On  the  other  hand,  the  danger  exists  that  the  demand  for  even  more 
input  data  causes  an  increase  in  time  and  energy  for  the  creation  of  simulation  models. 
This  paper  presents  a  model  for  the  integration  of  manpower-related  information  into 
production  planning.  The  paper  deals  with  the  question  to  what  extent  expanded  plan¬ 
ning  data  can  reduce  the  time  and  energy  for  collecting  data  and  how  simulation  models 
can  thus  be  prepared  more  quickly  and  more  efficiently. 

Introduction 

It  is  common  knowledge  that  simulation  cannot  replace  planning.  But,  simulation  is 
based  on  planning  results  as  well  as  on  the  establishment  of  the  necessary  technical  com¬ 
ponents  for  a  production  process  and  their  cooperation.  Simulation  itself  is  not  usable  for 
optimization,  but  it  can  help  to  achieve  an  optimal  output  of  the  system  with  regard  to 
one  or  several  goals.  Planning  and  simulation  must  work  together  within  iterative  steps 
for  that  purpose. 

Simulation  and  planning  tools  in  work  planning  support  the  high  resolution  modeling  of 
technical  systems  and  its  control.  Systems  and  processes  in  scheduling  and  work  plan¬ 
ning  make  use  of  the  available  resources  of  a  system  with  high  economy  and  minimal 
costs  to  manufacturing  of  products.  The  use  of  simulation  in  planning  and  logistics  aims 
basically  at  the  planning  stage  and  the  possibilities  to  changes  in  it.  Presently,  manufac¬ 
turing  simulation  and  planning  tools  allow  primarily  statements  about  the  economy  and 
effectiveness  of  the  considered  systems. 

Besides,  processes  in  production  and  logistics  are  at  this  time  characterized  by  changed 
work  tasks  the  workers  face  in  the  course  of  increasing  automation,  and  changed  struc¬ 
tures  in  labor  organization.  Workers  more  and  more  assume  monitoring,  checking  and 
controlling  tasks  by  virtue  of  the  increasing  automation.  Changed  structures  in  labor  or¬ 
ganization  require  the  workers  to  assume  tasks  which  have  predominantly  planning  and 
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organizing  character. 

In  the  field  of  scheduling  and  work  planning,  manpower  is  often  only  modeled  as  a  re¬ 
source,  like  machines  or  other  equipment  components.  In  planning  and  simulation,  man¬ 
power  is  available/not  available  and 
suitable  for  a  certain  task  or  not.  In 
fact,  man,  with  its  creativity  and 
flexibility,  is  one  of  the  most  impor¬ 
tant  elements  of  production  systems. 

And  thus  health  risks  can  jeopardize 
the  economy  of  production  systems, 
if  man  is  considered  as  a  subsystem. 

Because  of  missing  manpower-related 

information  in  the  planning  process,  j  .  Scopes  in  the  operation  scheduling 

it  is  not  possible  in  an  existing  manu¬ 
facturing  system  to  avoid  dangers  which  are  caused  by  current  equipment.  Especially 
dangers  to  the  workers,  caused  by  the  use  of  dangerous  materials  or  manufacturing 
methods  as  well  as  dangers  by  the  combination  of  material,  machine,  manufacturing 
methods  and  work  place,  cannot  be  removed.  In  the  end  of  work  planning  the  operation¬ 
ally  available  resources  are  tied  efficiently.  Also  the  room  to  maneuver  in  making  a  deci¬ 
sion  is  limited  from  the  view  of  manpower  studies.  So  it  would  be  better  to  evaluate  pro¬ 
cesses  in  their  planning  state.  At  a  later  stage  such  room  to  maneuver  will  be  very  re¬ 
stricted.  Based  on  this  knowledge,  manpower-  related  information  should  be  included 
into  the  processes  and  systems  of  production  planning. 

Background 

For  the  economic  scheduling  of  work  and,  at  the  same  time,  preventive  minimization  of 
strains  and  dangers  workers  could  be  exposed  to,  it  is  not  only  useful  but  necessary  to 
integrate  safety  aspects  into  the  planning  process.  The  following  extracts  from  the  Ger¬ 
man  Industrial  Safety  Act,  of  August  21,  1996,  underline  this  fact.  They  read: 


j ...  §3  Basic  duties  of  the  employer: 

(1)  The  employer  shall  be  obligated  to  take  measures  for  industrial  safety,  always  taking  into  account  those 
circumstances  which  have  an  impact  on  the  workers’ safety  and  health  ... 

...  §4  General  rules: 

(1)  Work  has  to  be  planned  so  as  to  best  possibly  avoid  any  dangers  to  life  and  health  and  minimize  any 
remaining  risks ... 

...  §5  Assessment  of  working  conditions 

(1)  By  assessing  the  risks  connected  with  the  employees’  work,  the  employer  is  to  determine  which  labor 
protection  measures  are  required. 

(3)  A  risk  can  particularly  be  caused  by 

...  the  design,  the  choice  and  the  use  of  working  tools,  in  particular  of  materials,  machines,  equipment  and 
devices  and  the  utilization  thereof, _ 
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...  the  scheduling  of  work  and  production  procedures,  work  sequences... 


Figure  2:  Extracts  from  the  German  Industrial  Safety  Act,  of  August  21,  1996 

The  preparation  of  any  factoiy  work  is  particularly  influential  in  the  scheduling  of  work 
and  production  processes.  Here  the  integration  of  any  information  on  risks  and  strains  is 
to  contribute  to  an  increased  planning  safety  regarding  aspects  of  operational  economy 
and  labor  protection.  The  analysis,  assessment  and  planning  mechanisms  at  an  early 
stage  of  the  planning  process  shall  constitute  a  considerable  contribution  in  order  to  put 
the  often  demanded  condition  to  "take  prospective,  instead  of  corrective  measures"  [7] 
into  operational  practice  when  planning  products  and  work  processes. 

The  action 

Within  the  framework  of  the  government  funded  GIPS  project  [German  abbreviation  for: 

Integration  of  Information  on  Strains 
into  Planning  Systems],  carried  out 
by  the  University  of  Magdeburg 
together  with  the  Dresden  Univer¬ 
sity  of  Technology,  an  approach  was 
brought  into  being  for  the  integra¬ 
tion  of  manpower-related  informa¬ 
tion  into  the  work  planning.  As  cri¬ 
teria  for  the  selection  of  manpower- 
related  information  to  be  integrated 
into  the  work  planning  served  the 
Figure  3:  Scheme  for  the  expansion  of  work  labor- scientific  evaluation  proce- 
planning  dures  BAB  -  [German  abbreviation 

for:  Evaluation  of  Workloads]  [3] 
and  SIA  -  Security  Analysis  according  to  NOHL  [4],  Starting  point  of  the  efforts  was  the 
systemization  of  the  data  and  information,  collected  and  archived  in  the  computer-aided 
process  of  production  preparation,  pertaining  to  the  system  components  Technology  - 
Organization  -  Personnel  (T-O-P)  and  the  selection  of  operationally  relevant  structures. 
On  the  basis  of  such  systemization,  the  planning  stage  of  the  process  of  work  preparation 
was  modeled  with  the  help  of  computer  technology.  This  model  is  based  upon  simplified 
and  mostly  generally  applicable  planning  data  and  data  structures.  The  model  serves  as 
basis  for  the  gradual  linking  of  information  on  risks  and  strains  with  the  classical  data 
structures  from  the  planning  section,  and  their  assignment  to  the  diverse  planning  stages. 
In  the  following  we  want  to  describe  our  approach  for  the  integration  of  manpower- 
related  information  into  the  work  planning.  This  approach  will  show  the  possibilities 
which  are  brought  about  for  the  depiction  of  risks  to  safety,  on  the  one  hand,  the  workers 
are  exposed  to  and,  on  the  other  hand,  for  the  description,  in  planning  and  simulation,  of 
organizational  sequences. 

The  starting  points 

Data  relating  to  the  technical  components  which  are  archived  in  the  planning  form  the 
basis  for  modeling  and  shaping  of  the  production  processes.  The  joint  action  of  the  tech¬ 
nical  components  and  the  sequence  of  steps  of  production  are  established  in  work  plans. 
The  work  plans,  prepared  independently  on  any  incoming  order  volume,  determine  a 
production  task  with  regard  to  the  required  technical  resources  and  the  technological 
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course  of  actions.  Work  plans,  order  and  variable  control  strategies  do,  as  ever,  deliver 
the  most  important  input  data  for  the  planning  and  shaping  of  production  processes,  on 
the  one  hand,  and  for  simulation  studies,  on  the  other  hand.  And,  work  plans  determine 
indirectly  the  actions  to  be  taken  by  the  workers.  The  collection  and  sorting  of  man¬ 
power-related  data  on  the  basis  of  work  plans  is  therefore  regarded  a  good  possibility  for 
the  integration  into  the  planning  process.  A  schematic  depiction  of  strain  characteristics 
allows  to  compare  diverse  work  traces  under  aspects  of  strains.  On  the  other  hand,  the 
data  pertaining  to  defined  periods  of  planning  will  provide  recommendations  and  re¬ 
quirements  for  the  production  planning  and  work  control. 

Departing  from  an  investigation  of  cause-and-effect-relationships  of  selected  safety¬ 
relevant  factors,  a  database- aided  model  was  developed  which  assigns  information  about 
strains  and  risks  to  the  conventional  planning  information  and,  furthermore,  uses  existing 
data  structures  in  the  planning,  modifying  them  so  that  it  is  possible  to  describe  human 
actions  and  the  effects  thereof.  Figure  4  shows  the  elaborated  scheme  regarding  the  inte¬ 
gration  of  safety-relevant  information  on  the  basis  of  those  decisions  which  are  taken 
within  work  planning,  and  of  the  potential  influences  which  arise  for  the  layout  of 
planned  work  traces. 


Archiving  of  basic  data 

Preparation  of  work  plans 

Work  control 

1  Information  conventionally  archived  in  the  planning  levels:  j 

-  Work  tools 

-  Objects  to  be  worked  on 

-  Production  techniques 

Partial  work  processes,  work  proc¬ 
esses,  work  plans  -  product-related, 
independent  on  orders  with  decisions 
regarding  work  tools,  production 
techniques,  objects  to  be  worked  on 

Production  plans  -  dependent 
on  orders,  related  to  structures 
with  decisions  regarding 
orders,  deadlines,  and  capaci¬ 
ties 

Trace  plans  dependent  on 

order,  related  to  manpower 

with  decisions  regarding 
distribution  of  work  and 
production  sequences 

1  Expansion  by  action^related  structures  and  data:  I 

-  Workplaces 
-Staff 

Partial  work  processes 
manpower- related;  work  processes  and 
work  plans  technical-organi-zational, 
and  ca'ganizational  level 

Work  tasks 

dependent  on  orders,  related  to 
structures 

Work  tasks 

dependent  on  orders,  related 
to  manpower 

1  Expansion  by  safety-relevant  data  (on  the  basis  of  analyses  of  strains  and  risks  according  to  BAB  and  SIA)  I 

-  Surrounding  influences 

-  Materials  and  limits 

-  Laws  and  regulations 

-  Standards  as  to  strains  and  risks,  planning  recommendations,  safety 
recommendations 

manpower-related  stan¬ 
dards  as  to  strains  and  risks 
as  decision-making  aid 

Figure  4:  Scheme  showing  the  integration  of  safety-relevant  aspects  into  work  prepara¬ 
tion 
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Analysis  and  Evaluation  of  Strains  and  Risks 

As  mentioned  before,  the  manpower-related  information  to  be  integrated  is  selected  in 
the  GIPS  project  following  the  evaluation  procedures  BAB  [3]  and  SIA  [4].  These  proce¬ 
dures  are  based  on  the  analysis 
of  a  number  of  single  character¬ 
istics  that  are  collected  within  a 
work  system  by  measurements, 
descriptions,  analyses,  and  simi¬ 
lar. 

The  two  procedures  examine  and 
assess  factors  that  are  brought 
about  by  the  co-acting  T-O-P 
components  of  a  given  work 
system  and  that  have  an  impact 
on  the  worker.  The  emphasis  of 
the  two  procedures  is  put  on  the 
safe  shaping  of  both  sequence- 
dependent,  ergonomic  characte¬ 
ristics  (BAB)  and  sequence- 
dependent,  technical  characteristics  (SIA)  of  any  studied  work  system. 

For  a  model  to  include  such  manpower-related  information  in  the  planning  process,  like 
shown  in,  different  levels  are  considered: 

1  the  description  level,  which  consists  of  the  expansion  of  the  operational  da¬ 
tabase, 

2  the  analysis  level  and  the  assessment  level,  which  constitute  an  expansion  of 
the  planning  methods.  There  must  be  integrated  mechanisms  for  analysis  and 
assessment  of  expanded  planning  data.  These  mechanisms  must  correspond  to 
already  existing  planning  mechanisms. 

3  the  modification  level  to  produce  examples  of  planning  alternatives. 

In  order  to  fully  exploit  the  possibilities  and  alternatives  arising  therefrom,  the  levels 
must  be  integrated  step  by  step  into  each  single  phase  of  the  work  planning  process. 

On  the  basis  of  the  described  BAB  and  SLA  approaches,  individual  factors  of  risks  and 
strains  were  selected  and  integrated  into  the  planning  process.  In  the  following  we  de¬ 
scribe  the  approach  for  the  gradual  integration  of  safety-relevant  information  into  the 
planning  process,  making  use  of  the  planning  levels  described  in  Figure  4. 

The  approach 

Conventional  computer-aided  planning  systems,  the  functional  scope  of  which  we  leaned 
on  when  preparing  the  model,  provide  functions  for  the  compilation  and  archiving  of 
basic  data  of  a  given  work  system,  such  as  production  procedures,  objects  to  be  worked 
on  and  work  tools.  Procedures  like  those  of  BAB  or  SIA  are  based  upon  the  analysis  and 
assessment  of  factors  which  are  collected  in  real  traces.  Thus  the  data  collected  in  such 
procedures  characterize  and  assess  a  situation  arising  from  the  co-action  of  all  compo¬ 
nents  in  any  studied  work  system.  However,  in  order  to  convey  the  manpower-related 
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information  obtained  into  expanded  work  planning  systems  and  processes,  it  is  necessary 
to  classify  the  cause-and-effect-relations  pertaining  thereto.  Then  the  results  obtained 
from  human  work  analyses  can  be  assigned  to  the  corresponding  planning  data.  A  meas¬ 
ured  intensity  of  light  is  an  example  of  an  attribute  of  Ae  workplace.  Information  con¬ 
cerning  the  dangers  of  toxic  or  corrosive  substances  are  assigned  to  the  respective  mate¬ 
rials.  Machines  may  have  corresponding  information  which  refers  to  danger  spots.  The 
description  level  integrates  the  experience  of  manpower  studies  into  planning  systems. 
Apart  from  the  so-called  "measured  data",  also  the  "required  data"  resulting  from  laws, 
regulations  and  guidelines  for  the  use  of  the  resources,  must  be  capable  of  being  assigned 
to  the  corresponding  planning  data.  So  guidelines,  instructions  and  maximum/minimum 
values  for  harmful  substances  were  prepared,  as  an  example.  The  prepared  model  pro¬ 
vides  the  user  with  description  aids,  so-called  bridge  examples,  for  the  described  assign¬ 
ment. 

Computer-aided  instruments,  such  as  CAP  systems,  accompany  the  phase  of  work  plan 
preparation.  They  contain  functions  for  the  compilation  and  archiving  of  work  plans. 
Work  plans  which  describe  the  production  steps  towards  a  given  product  have  so  far 
been  conventionally  prepared  through  step-by-step  assembling  of 

•  partial  work  processes,  through  the  assembling  of  production  techniques, 
work  tools,  and  the  object  to  be  worked  on; 

•  work  processes,  as  the  result  of  1-n  partial  work  processes;  and 

•  work  plans,  as  the  result  of  1-n  work  processes. 

In  the  phase  of  work  plan  preparation,  thus  the  safety-relevant  values  which  are  specific 
for  the  work  system,  and  the  required  values  meet  which  have  been  assigned  to  the  level 
of  base  data  collection  and  archiving.  Since  the  entering  into  ejfectiveness  of  strains  and 
risks,  however,  prerequisites  human  actions,  the  depiction  of  human  activities  within  the 
stage  of  work  plan  preparation  is  necessary.  Only  on  that  basis  is  it  possible  to  integrate 
adequate  analysis  and  assessment  mechanisms  into  the  individual  planning  steps.  The 
mechanisms  must  correspond  to  already  existing  planning  mechanisms.  It  was  necessary 
to  identify  planning  steps  with  the  utmost  decision  room,  and  so  safety  and  stress  inten¬ 
sity  data  could  be  coupled  efficiently  with  traditional  planning  data. 

As  mentioned  above,  the  preparation  of  work  plans  brings  about  indirectly  the  determi¬ 
nation  of  those  actions  the  workers  are  required  to  cany  out.  In  the  prepared  model  the 
human  actions  are  integrated  into  the  level  of  preparation  of  work  plans,  starting  from  the 
description  of  the  partial  steps  of  production.  Apart  from  the  usual  production-oriented 
partial  processes  which  describe  the  single  production  steps  towards  a  product,  so-called- 
manpower-related  partial  steps  of  production  are  integrated  into  planning.  Manpower- 
related  partial  steps  of  production  describe  a  worker’s  partial  activities  which  do  not 
modify  the  product  as  such.  With  regard  to  their  functions  we  distinguish  between  trans¬ 
portation  and  load  manipulation  activities,  such  as  delivery  of  parts  and  relocation,  and 
activities  of  examination,  measuring,  etc.  Information  on  strains  and  risks  which  result 
directly  from  the  partial  activity,  such  as  physical  strain  factors,  may  be  directly  assigned 
to  partial  processes.  The  strains  and  risks  resulting  from  partial  work  processes  in  which 
manpower  is  involved  are  equal,  unless  they  refer  to  the  situation  in  a  special  work  sys¬ 
tem.  The  carrying  of  a  box  of  40  pounds  over  a  certain  distance  will  be,  as  an  example,  as 
much  burdensome  for  a  worker  of  firm  A,  as  it  will  be  for  a  worker  of  firm  B.  Departing 
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from  this  knowledge,  a  database  of  ergonomically  compiled  manpower-related  activities 
can  be  prepared.  The  GIPS  project  is  about  to  create  first  bases  therefor. 

A  connection  between  the  product-related  trace  steps  and  the  manpower-related  activities 
can  be  found  with  the  preparation  of  work  processes,  where  manpower-related  activities 
are  assigned  to  the  several  production  steps  leading  to  a  certain  product.  Within  one  pro¬ 
duction  step,  we  distinguish  between 

•  main  activities  which  describe  the  processing  of  a  product  and  which  bind 
manpower  up  to  100%,  dependent  on  the  level  of  automation; 

•  preparing  and  follow-up  activities  which  are  necessary  to  prepare,  or  assess 
or  treat  afterwards  the  main  activity,  binding  manpower  up  to  100%;  and 

•  accompanying  activities  which  a  worker  has  to  carry  out  parallel  to  the  main 
activity. 


The  introduction  of  manpower-related  activities  in  the  work  process  and  its  time  share 
allows  here  a  first  assessment,  with  the  BAB  and  SIA  approaches,  of  strain  and  risk  fac¬ 
tors  workers  are  exposed  to.  For 
the  description  of  manpower- 
related  work  courses  within  the 
planning  process,  the  work  proc¬ 
esses  in  the  model  are  classified 
into  technical  and  organizational 
processes.  While  technical  work 
processes  subsume  all  necessary 
partial  processes  relating  to  a  spe¬ 
cific  production  step  toward  a 
product,  the  class  of  the  organiza¬ 
tional  work  processes  serves  the 
description  of  actions  which,  for 
their  part,  serve  to  implement  the 
production  process.  Processes  of 
this  class  are,  for  example,  proc¬ 
esses  of  changing  tools  and  of  transportation,  as  well  as  planning  and  organizational  pro¬ 
cesses.  They  serve  to  define  organizational  actions  both  with  regard  to  time  and  to  the 
demands  posed  to  the  workers.  Organizational  work  processes  are  of  special  interest,  as 
an  example,  for  the  contemplation  of  psychological  strains.  Nature  and  number  of  the 
actions  to  be  carried  out  can  be  used,  among  others,  for  examining  responsibilities  and 
demands  on  quality  and  concentration. 

In  strict  observance  of  the  above  approach  we  can  distinguish  three  types  of  work  plans: 

•  technical  work  plans  -  describe  the  course  of  production  towards  a  specific 
part; 

•  technical  -organizational  work  plans  -  describe  recurring  organizational, 
however,  technically  determined  courses  within  the  work  process,  such  as 
maintenance  cycles,  changing  of  the  tools  and  transportation  processes.  Such 
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Figure  6:  Extended  data  structures  in  work  plan- 
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processes  are  required  for  maintaining  the  technical  process; 

•  organizational  work  plans  -  describe  recurring,  and  purely  organizationally 
determined  processes  within  the  whole  work  process,  e.g.  planning  cycles 
and,  possibly,  rules  providing  for  intervals  and  shifts.  They  allow  the  descrip¬ 
tion  of  sequences  which  are  of  informative,  planning  and  controlling  charac¬ 
ter. 

Like  at  every  planning  stage  of  the  developed  model,  relevant  risk  and  strain  factors  are 
also  assessed  when  preparing  the  work  plans. 

By  that  stage  of  the  work  preparation  process,  the  most  important  bases  for  a  detailed 
description  of  human  work  within  the  framework  of  work  planning  have  been  estab¬ 
lished.  Apart  from  the  activities  related  to  the  fabrication  of  a  product,  in  particular  the 
monitoring,  controlling  and  similar  activities  are  described  which  are  common  in  highly 
automated  production  processes.  The  thus  established  link-up  of  technical  work  plans 
with  activity  sequences  create  multiple  possibilities  both  in  the  next  steps  of  the  planning 
process  and  in  simulation. 

During  the  phase  of  preparation  of  production  plans  the  incoming  orders  are  inserted 
ficticiously  into  a  system.  With  the  help  of  the  expanded  work  plan  data  it  will  be  possi¬ 
ble  to  depict  not  only  technical  work  plans  in  connection  with  orders  at  a  time  axis 
scheme,  but  also  the  work  courses  which  maintain  and  control  the  production  process.  If 
such  technical-organizational  and  organizational  courses  of  work  are  understood  as  or¬ 
ders  for  a  given  system,  it  will  also  be  possible  to  depict  the  sequence  of  necessary  ac¬ 
tions  to  be  taken  by  the  workers,  in  order  to  make  it  possible  not  only  to  check  the  tech¬ 
nical  "feasibility"  and  "economic  efficiency",  but  to  see  strains  and  risks  which  result,  as 


Figure  7:  Expanded  functions  in  work  planning 


an  example,  from  simultaneously  running  production  and  work  courses. 

The  level  of  work  control  serves  the  scheduling  of  a  given  work  process  for  a  given  sec¬ 
tion  and  within  a  defined  time  period.  In  doing  so,  the  tasks  being  at  the  agenda  are  as¬ 
signed  to  the  worker  or  the  course  is  organized  with  regard  to  the  workers.  In  this  situa- 
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tion,  concrete  characteristics  of  persons,  like  qualifications,  age  and  sex,  etc.  meet  with 
the  requirements  of  the  work  tasks.  Herein,  the  model  gives  corresponding  assistance  in 
the  assessment  of  strains  and  risks  and  also  in  the  decision-finding.  If  safety  risks  be¬ 
come  visible  within  the  course  of  planning,  planning  alternatives  have  to  be  searched  for. 
For  the  elimination  of  insufficiencies,  the  reiteration  of  planning  steps  will  be  required  in 
the  first  place. 
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Figure  8:  Depiction  of  the  capacity  utilization  of  workers  at  workplaces 

(sirrppn  vhnf)  . 


ments.  Safety  and  danger  prevention  recommendations  are  added  to  the  operational  in- 
stmctions  for  the  worker  (wearing  of  ear  protectors  required,  etc.). 
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in  production  and  logistics  the  human  behavior  frequently  becomes  the  critical  point,  a 
"bottleneck". 


The  developed,  computer-aided  model,  with  the  help  of  the  combined  technical  and 
manpower-related  processes  in  the  planning,  establishes  the  basis  for  a  new  quality  of  the 
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Figure  10:  Extract  from  a  risk  and  strain  scheme  (screen  shot) 


and  additional  work  plans  and  work  plan  data  allow  the  consideration  of  manpower- 
related  effects  in  simulation.  The  developed  model  uses  a  simulation  model  for  the  analy¬ 
sis  and  assessment  of  simultaneously  running  processes.  The  simulation  generates  a  trace 
file  which  serves  to  investigate  economic  questions  and  manpower-related  effects.  In  the 
metal-processing  industry,  first  successful  results  have  been  achieved  with  an  application 
under  the  GIPS  project.  Further  applications,  such  as  in  the  glass  and  ceramics  industry, 
serve  to  examine  their  acceptability  among  industrial  users. 

Conclusions 

In  summary  the  described  approach  seems  to  have  succeeded,  always  taking  into  account 
action-related  information  regarding  work  planning,  in  developing  a  basis  for  the  de¬ 
scription,  modeling  and  simulation  of  workers,  their  activities  and  manpower-related 
interactions  with  technology  and  organization  in  connection  with  the  production  proc¬ 
esses.  As  before,  work  planning  data  are  being  used  as  input  data  in  simulation  studies. 
In  the  planning  process,  the  developed  model  provides  a  comprehensive  description  of 
the  workers’  activities  in  connection  with  the  fabrication  of  the  products,  on  the  one 
hand,  and  the  maintenance  of  the  course  of  production,  on  the  other  hand.  Statutory 
regulations  in  Germany  concerning  the  observance  of  industrial  safety  and  health  protec¬ 
tion  force  already  today  to  integrate  manpower-related  data  into  the  planning  process.  In 
light  of  the  broad  scope  of  application  of  the  method  presented  in  this  paper  concerning 
the  depiction  of  manpower-related  factors  in  the  planning  process  and  a  comprehensive 
database  of  assessed  activities  (Figure  7),  it  is  likely  that  the  time  and  energy  spent  on  the 
collection  of  data  for  the  simulation  can  be  minimized. 
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Abstract 

The  simulation  of  physiological  processes  will  most  be  done  on  computers.  For  some 
different  purposes  it  is  ingenious  to  have  also  (hydro-)  mechanic  (inter- )  actions.  Two  of 
these  fundamental  ideas  are  the  roots  of  our  computercontroUed  electrohydraulic 
phantom.  This  phantom  can  be  used  for  quality  assurance  and  -control  of  ultrasound 
diagnostic  equipment  but  for  the  education  of  medical  practitioners  on  such  equipment 
too.  An  in  vitro  Doppler  ultrasound  phantom  has  been  invented  to  simulate  physiological 
and  non-physiological  blood  flow  waveforms  (in  the  sense  of  peripheral  hemodynamics). 
The  teacher  may  manipulate  the  flow  waveforms  and  practice  exams  to  his  students 
working  on  the  phantom.  For  calibrating  and  testing  Doppler-Ultrasound-Equipment  it  is 
not  necessary  to  generate  physiological  waveforms  but  a  reproducible  continuous 
bloodflow. 

Introduction 

The  Doppler  ultrasound  diagnosis  is  of  highly  diagnostic  importance  for  medical 
investigators  because  of  its  nonivasive  measurements  of  intracorporal  bloodflow 
parameters.  The  investigator  can  see  the  acoustic  (ultrasound)  picture  of  the  tissue  in  the 
region  of  interest  and  at  the  same  time  he  can  see  the  speed  and  flowdirection  of  the 
blood  visualized  by  colorcoding.  These  flowpattems  are  typical  for  eveiy  part  of  a 
human  blood  vessel,  so  they  are  called  „fingerprints“.  There  are  two  fundamental  sources 
of  errors: 

•  The  system  error  of  the  equipment  (up  to  the  power  of  10!), 

•  The  errors  caused  by  the  operator  by  using  the  applicator  or  the  settings  of  the 
equipment  in  a  deficient  way. 

The  errors  wouldn’t  be  detected  by  the  untrained  medicine  wo/man.  Putting  the 
applicator  in  a  „wrong“  angel  on  the  skin  is  very  simple  and  will  cause  erroneous  speed 
displays  due  to  the  Doppler-formula.  The  angel  between  blood  vessel  and  the  sound  wave 
influences  the  result  by  the  cosine.  Also  the  region  of  interest  (ROI)  may  not  be  in  the 
center  of  the  blood  vessel.  Due  to  the  parabolic  flow  profile  there  are  the  next  errors 
possible. 
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The  hardware 

The  phantom-equipment  (see  fig.  1)  consists  of  a  PC,  a  steppermotor-unit  mounted  to  a 
mohnopump  (wormpump)  and  a  cardiac  rubber  tissue  phantom  (ATS  Model  #523A  with 
4  different  tube  (vessel)  diameters).  The  pump  doesn't  need  self-lubricant  fluid  nor 
damages  the  particles  of  the  blood  (or  it's  blood-mimicking  substitute)  like  gear  pumps 
do.  In  our  circuit  made  of  rigid  tubing  (PVC,  to  avoid  the  Windkesseleffect),  a  blood 
mimicking  fluid  from  ATS  (Doppler  Test  Fluid  #707-2G)  is  pumped  either  producing  a 
continuous  flow  or  showing  a  physiological  or  non  physiological  pulsflow,  even  with  any 
kind  of  reflow.  The  physiological  Windkesseleffect  will  be  simulated  by  the  (fuzzy-) 
controlling  computer  (PC). 


Figure  1:  The  hardware-circuit 


How  to  work  with  the  phantom 

The  velocity-flow-curve  is  to  be  chosen  during  a  graphic  oriented  dialogue  on  the  PC 
under  MS-Windows.  Typical  "fingerprints'*  (see  fig  2)  are  stored  in  a  database  on  the  PC- 
harddisk.  They  are  „calibrated“  for  one  complete  cycle  of  the  heart.  The  user  will  be  able 
to  construct  own  flow-curves  or  edit  one  from  the  database  by  using  the  system’s  mouse 
in  a  simple  graphic-editor.  It  is  no  problem  for  the  user  constructing  non-physiological 
flow-profiles  too  for  checking  the  maximum  rates  of  the  diagnostic-system,  for  example. 


475 


pressure 


Parameters  like  heartfrequency,  tubediameter  and  flowvolume  are  introduced  by  editing 
the  predefined  values  in  a  pull  down  menu  (see  fig  3). 


Figure  3:  Pull  down  menu  for  Parameters 


The  parameters  for  constant  flow  are  typed  in  using  the  pulldown  menus  only,  because  a 
graphic  interface  makes  no  sense  therefor.  When  editing  is  already  done,  the  „trainer“ 
should  use  a  special  menu-entry  for  the  degassing  of  the  system.  When  the  circuit  has 
been  unused  for  some  days  or  the  tube  diameter  has  been  changed,  there  are  possibly  air- 
bubbles  in  the  tubesystem.  The  airbubbles  can  be  removed  by  pumping  the  blood 
mimicking  fluid  for  some  minutes  in  alternating  directions  through  the  circuit,  leaving 
the  bubbles  in  the  fluid  reservoir.  Than,  when  the  system  is  ready  and  running  its 
programmed  flowsimulation  through  the  tubes,  the  „student“  should  hold  the  applicator 
onto  the  tissue-mimicking  part  of  the  phantom.  On  his  diagnostic  equipment  monitor  he 
can  see  the  influence  of  the  angle  he  holds  the  applicator  or  of  the  different  machine 
settings.  It  is  the  way  the  „teacher“  checks  what  the  students  diagnose  -  without 
bothering  real  patients. 
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Other  simulations 

By  the  way:  we  also  programmed  a  simple  simulator  on  a  PC  for  CBT  (computer  based 
training).  Alike  the  above  shown  electrohydraulic  phantom  the  medical  wo/man  will  be 
introduced  and  trained  using  Doppler-Ultrasound-Diagnosis-Equipment.  We  took  a  data- 
glove  as  a  three-dimensional  input  device.  On  the  graphic  output  device  (video-monitor) 
a  laying  body  is  shown  for  the  investigation.  The  medical  practitioner  is  able  to  work 
with  a  simulated  ultrasound  device  (like  a  mousepointer)  on  this  represented  virtual 
body.  The  trainee  can  see  the  actual  bloodflow  pattern  (in  a  window)  of  the  region  he  is 
positioning  his  virtual  device  on.  When  he  dives  the  virtual  applicator  into  the  body 
instead  of  gliding  on  the  surface  (skin  of  the  body)  a  voice  from  the  PC-speaker  is 
protesting.  The  typical  sound  generation  of  the  diagnostic  equipment  is  not  yet 
implemented  due  to  a  shortage  of  interest  of  the  medical  wo/men. 

Outlook 

Another  computercontrolled  electrohydraulic  project  is  under  process  in  cooperation  with 
the  university  hospital  of  Hamburg.  The  bloodflow  of  the  leftventricular  heart  will  be 
simulated  for  the  validation  of  3-dimensional  intraoesophagial  ultrasound  scanners.  Also 
possible  are  examinations  of  artificial  heartvalves. 

Conclusions 

This  blood  flow  phantom  would  be  valuable  during  the  education  of  ultrasound  Doppler 
diagnosis  for  physicists.  The  student  puts  the  applicator  of  the  ultrasound  diagnostic 
equipment  on  the  tissue-phantom  an  will  be  trained  by  a  teacher  to  achieve  and  identify 
the  simulated  flow  patterns  and  conclude  a  diagnosis.  The  teacher  is  able  to  chose  the 
flow-patterns  from  a  database  or  to  generate  special  patterns  in  a  graphic-oriented  dialog 
on  the  PC. 

References 

Dolk,  Andreas:  Entwicklung  einer  grafisch  /  alphanumerischen  Benutzeroberflache  fur 
ein  Ultraschall-Phantom  mit  Ubergabe  der  Daten  an  einen  Sekundarrechner, 
Diplomarbeit  1992,  issued  and  attended  on  by  Ulrich  G.  Schaarschmidt 

Cordes,  Bemd:  Entwicklung  einer  interaktiven  grafischen  Benutzeroberflache  fiir  MS- 
Windows,  Diplomarbeit  1993,  issued  and  attended  on  by  Ulrich  G. 

Schaarschmidt 

Simon,  Ingo:  Regelung  eines  Ultraschall-BlutfluBphantoms  mit  Hilfe  der  Fuz^-Technik, 
Studienarbeit  1993,  issued  and  attended  on  by  Ulrich  G,  Schaarschmidt 

Simon,  Ingo:  Optimierung  der  Regeleigenschaften  und  Vergleichsmessungen  zur 

Kalibrierung  eines  BlutfluBphantoms,  Diplomarbeit  1993,  issued  and  attended  on 
by  Ulrich  G.  Schaarschmidt 

Kirchg^ner,  Ansgar :  Entwurf  eines  multimediabasierenden,  interaktiven 

Ausbildungssystems  unter  Windows,  Diplomarbeit  1994,  issued  and  attended  on 
by  Ulrich  G.  Schaarschmidt 

Schaarschmidt,  Ulrich  G. :  Phantom  zur  Kalibrierung  von  Ultraschall-Doppler-Geraten 
und  zur  Simulation  physiologischer  Eigenschaften  in  der  hamodynamischen 
Diagnostik,  Thesis  1994. 


477 


Perception-action  compatibility  and  eye-hand  dominance  in  using 

visually-displayed  information 


Walter  H.  Ehrenstein  and  Birgit  K  Amold-Schulz-Gahmen 

Institut  fUr  Arbeitsphysiologie 
University  of  Dortmund 
44139  Dortmund,  Germany 


Abstract 

The  efficient  use  of  visually- displayed  information  depends  largely  on  the  principle  of 
perception-action  compatibility  which  predicts  that  the  way  in  which  information  is 
perceptually  provided  predetermines  the  accuracy  and  speed  of  the  operations  required. 
This  principle  was  studied  in  connection  with  the  respective  eye-hand  dominance  of  the 
operator.  Subjects  with  pronounced  dominances  of  their  eye  and  hand  were  selected 
according  to  a  test  of  ocular  dominance  and  two  laterality  questionnaires.  Their  task  was 
to  react  to  horizontal  visual  motion  which  started  at  the  left  or  right  side  of  a  computer 
screen  by  use  of  a  static  or  a  dynamic  response  tool  (key  vs.  joystick).  Static  or  dynamic 
stimulus  cues  (onset  position  vs.  motion  direction)  were  assigned  to  static  or  dynamic 
responses  in  a  compatible  (e.g.,  left  or  leftward  responses  to  left  or  leftward  stimuli),  or 
incompatible  (left  or  leftward  responses  to  right  or  rightward  stimuli)  way.  Overall,  right¬ 
handers  responded  faster  than  left-handers  and  right-eyed  persons  tended  to  be  faster 
than  left-eyed  persons.  Cue-tool  compatibility  and  spatial  compatibility  effects  interacted 
with  handedness,  but  not  with  eye  dominance,  in  that  the  difference  between  left-  and 
right-handers  increased  under  incompatible  conditions;  in  other  words,  the  disadvantage 
of  left-handers  is  significantly  reduced  under  compatible  conditions.  Thus,  an  ergonomic 
design  that  improves  the  workplace  design  just  on  the  base  of  the  compatibility  principle 
alone  might  help  to  ease  the  situation  of  left-handed  persons  who  seem  to  depend  above 
all  on  compatibility  conditions. 

Introduction 

Efficient  ways  of  displaying  information  have  been  a  major  concern  of  the  human  factors 
approach  since  its  early  days.  The  increased  complexity  of  today’s  dynanuc  systems 
entails  the  presentation  of  a  wealth  of  information  to  the  human  operator.  At  the  same 
time,  rapid  advances  in  computer  and  display  technology  have  increased  both  the 
capability  of  displaying  multi-element,  complex  information  and  the  freedom  to  select 
the  aspect  and  mode  of  presentation.  General  guidelines  for  the  design  of  complex  visual 
displays  should  include  the  following  items  (Gopher  &  Kimchi,  1989):  1.  Aspects  of  the 
represented  world  (e.g.,  a  complex  dynamic  system)  should  be  selectively  captured 
within  the  representing  world  (i.e.,  the  visual  display);  2.  The  selected  aspects  should  be 
represented  in  an  optimal  way;  and  3.  One  should  recognize  that  items  1  and  2  depend 
critically  upon  task  domain,  and  thus  require  a  coherent  analysis  of  demanded  actions  in 
relation  to  perceptual  information. 
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The  principle  of  compatibility 

Even  in  relatively  simple  tasks,  like  that  of  pressing  a  key  in  response  to  a  light  flashed  at 
a  certain  location,  the  time  and  accuracy  of  the  operation  are  influenced  by  perception- 
action  (stimulus-response)  compatibility,  i.e,,  by  the  relationship  between  the  specific 
stimulus  and  response  sets  that  are  used  and  the  way  in  which  the  members  of  the 
stimulus  set  are  assigned  to  the  responses  (Proctor  &  Dutta,  1995).  Performance  tends  to 
be  faster  and  more  accurate  when  the  signal  and  an  operator’s  response  to  it  are 
perceptually  related,  e.g.,  when  both  are  located  in  the  same  part  of  the  operator’s  visual 
(left  or  right)  hemifield.  The  principle  of  compatibility  which  emerged  in  the  field  of 
human  factors  in  the  early  1950s  (e.g.,  Fitts  and  Seeger,  1953)  has  been  applied  since 
then  to  many  areas  such  as  spatial  coding,  man-machine  interactions,  and  optimal  design 
of  displays  or  keyboards  (see  Wickens,  1992;  Proctor  &  Van  Zandt,  1994,  for  reviews). 
The  principle  predicts  in  general  terms  that  the  way  in  which  information  is  presented 
predetermines  the  accuracy  and  speed  of  the  operations  required. 

Static  and  dynamic  stimulus-response  pairings 

Up  to  the  1990s,  static  stimuli  and  response  tools  have  almost  exclusively  dominated  the 
research  of  perception- action  compatibility  (Proctor  &  Reeve,  1990),  With  the  advent  of 
high-powered  computer  graphics  and  real-time  recording  of  responses,  moving  stimuli 
and/or  dynamic  response  tools  have  been  increasingly  employed  (e.g.,  Ehrenstein,  1994; 
Amold-Schulz-Galmen  &  Ehrenstein,  1995,  1996;  Arnold- Schulz-Gahmen  et  al.,  1996; 
Cavonius  et  al.,  1996;  Ehrenstein  et  al,  1996),  and  dynamic  accounts  (e.g.,  Grassberger- 
Procaccia  algorithms)  applied  to  the  field  of  compatibility  research  (Ganz  et  al.,  1996; 
Ehrenstein,  1997),  In  the  present  study,  dynamic  visual  stimuli  were  chosen,  to  which 
subjects  reacted  by  means  of  static  or  dynamic  response  tools  (keypress  or  deflection  of  a 
joystick). 

Eye-Hand  Dominance 

Any  ergonomic  application  of  perception- action  compatibility  would  be  incomplete  if  it 
ignored  the  laterality  or  dominance,  both  of  eye  and  hand,  of  a  given  operator.  About 
one  in  three  people  is  left-eyed,  about  one  in  ten  is  left-handed  (Nachson  et  al,  1983; 
Bourassa  et  al.,  1996).  As  there  are  3  times  more  left-eyed  than  left-handed  persons,  it 
follows  that  a  substantial  percentage  in  a  population  has  a  crossed  eye-hand  dominance, 
e.g.,  is  right-handed,  but  left-eyed.  In  their  monograph  on  handedness  and  manipulation 
of  work  tools,  Schmauder  and  Solf  (1992)  state  a  lack  of  verified  insights  concerning  the 
effects  and  requirements  of  handedness  in  human  ergonomics.  Even  less  is  known  about 
the  influence  of  eyedness  and  the  putative  interactive  influences  of  eyedness  and 
handedness  in  the  use  of  visually  displayed  information,  especially  with  respect  to  spatial 
compatibility.  This  means  that  the  present  study  is  entering  a  rather  new  field  of  research 
and  is  hence  of  a  rather  exploratory  nature. 

Aims  of  the  Present  Study 

In  the  present  study  we  asked  whether  the  speed  of  action  or  response  time  (RT)  depends 
on  the  respective  (i)  compatible  relationship  between  (static  or  dynamic)  stimulus  cues 
and  response  tools,  (ii)  the  compatible  spatial  assignment  of  responses  to  stimuli  (left  or 
right),  and  (iii)  eye-hand  dominance  of  the  operator  (in  the  sense  that  it  would  interact 
with  the  two  above  compatibility  effects). 
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Method 

Subjects 

The  pointing  test  by  Walls  (1951)  and  the  laterality  questionnaires  by  Oldfield  (1971) 
and  Coren  (1993),  in  their  German  adaptations  (Ehrenstein  &  Amold-Schulz-Gahmen, 
1997)  were  used  to  select  subjects  with  pronounced  dominances  (lateralization)  of  their 
eyes  and  hands  from  a  larger  sample.  Four  subjects,  matched  in  age  and  gender,  formed 
one  of  four  dominance  subgroups:  left-eyed,  left-handed  (LE-LH),  left-eyed,  right- 
handed  (LE-RH),  right-eyed,  left-handed  (RE-LH),  and  right-eyed,  right-handed  (RE- 
RH). 

Task 

These  four  groups  of  subjects  were  tested  in  a  choice-reaction  task  in  which  a  static  or 
dynamic  response  was  assigned  compatibly  or  incompatibly  to  either  the  onset  position 
or  motion  direction  of  a  moving  visual  stimulus  (see  Fig.  1).  The  stimulus  was  a  bright 
dot  that  appeared  suddenly  (after  a  randomly  selected  interval  of  600  -  900  ms  following 
a  warning  ‘beep’)  at  5  deg  left  or  right  of  fixation,  moving  leftward  or  rightward  at  2 
deg/s  for  1  s. 


Stimulus 


Onset  Position  Motion  Direction 


L  R  L  R  L  R 


static  cue 

dynamic  cue 

static 

response 

tool; 
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L  R 

_ CD _ 

dynamic 

response 

tool: 
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Figure  I:  Experimental  set-up:  Each  subject  participated  two  times  in  four  conditions 
(presented  in  separate  blocks)  in  which  either  stimulus  onset  position  (irrespective  of 
direction,  Conditions  J  and  2)  or  motion  direction  (irrespective  of  onset  position, 
Conditions  3  and  4)  was  the  relevant  cue.  The  response  was  given  by  static  (keypress) 
or  dynamic  (joystick)  tools.  Keypress  location  or  joystick  direction  was  assigned 
compatibly  (e.g.,  left  keypress  to  left  stimulus  onset)  or  incompatibly  (e.g.,  right 
keypress  to  left  onset  stimulus)  in  all  four  conditions. 


Each  subject  participated  in  eight  conditions  (presented  in  separate  blocks  of  trials)  in 
which  either  the  position  of  motion-onset  (irrespective  of  direction,  see  Fig.l,  conditions 
1  and  2),  or  the  direction  of  movement  (irrespective  of  position,  Fig.  1,  conditions  3  and 
4)  was  the  relevant  stimulus  cue.  The  response  was  static  (keypress)  or  dynamic  (joystick 
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deflection).  Key  location  or  response  direction  by  joystick  deflection  was  assigned 
compatibly  (e.g.,  the  right  key  was  pressed  or  the  joystick  was  moved  to  the  right  when 
the  stimulus  occurred  on  the  right)  or  incompatibly  (e.g.,  the  left  key  was  pressed  or  the 
joystick  was  deflected  to  the  left  when  the  stimulus  occurred  on  the  right). 

In  the  static  response  tool  condition  keys  at  the  left  and  right  positions  were  pressed  with 
the  left  or  right  index  finger,  respectively.  In  the  dynamic  response  tool  condition  the 
joystick  was  held  with  the  whole  (preferred  or  nonpreferred)  hand  (alternating  in 
successive  blocks  of  trials). 

Results 

Mean  RTs  were  analyzed  by  ANOVA  with  eyedness  and  handedness  as  between- 
subjects  factors  and  stimulus  cue  (onset  position  or  motion  direction),  response  tool  (key 
or  joystick),  and  compatibility  (compatible  or  incompatible  assignment  of  stimuh  to 
responses)  as  within-suhjtcis  factors. 

Eye-hand  dominance 

Overall,  right-handed  subjects  were  significantly  faster  than  left-handed  subjects  by 
41.2  ms  [F(l,12)  =  5.57,/)  =  .036].  Right-eyed  subjects  responded,  on  average,  15.2  ms 
faster  than  left-eyed  individuals,  but  this  difference  was  not  significant  [F(l,12)  =  0.83, 
p  =  .38],  and  eyedness  did  not  interact  with  any  other  factor,  hi  other  words,  we  found  no 
effect  of  eyedness  and  no  benefit  for  shared  eye-hand  dominance  (e.g.,  both  right-eyed 
and  right-handed  subjects)  over  crossed  eye-hand  dominance  (e.g.,  right-eyed,  but  left- 
handed  subjects). 

Cue-Tool  Compatibility  and  Handedness 

Responses  to  onset  position  were,  on  average,  faster  than  responses  to  motion  direction 
(351.9  vs,  442.7  ms,  respectively).  Interestingly,  stimulus  cue  interacted  with  both 
response  tool  and  handedness.  The  stimulus  cue  x  response  tool  interaction  [F(l,12)  = 
15.03,  p  =  .0022]  refers  to  the  fact  that  the  speed  of  reaction  to  motion  onset  depended 
on  the  response  mode,  i.e.  reactions  were  much  faster  with  keypress  (334.1  ms)  than 
joystick  (369.9  ms)  responses,  whereas  mean  RT.  to  motion  direction  did  not  depend  on 
the  response  mode  (442.2  v^-.  443.3  ms  for  keypress  vs,  joystick  responses). 

The  3- way  interaction  (stimulus  cue  x  response  tool  x  handedness  [F(l,12)  =  5.19,  p  = 
.042]  reflects  the  fact  that  right-handers  were  52.6  ms  faster  than  left-handers  when  the 
relevant  stimulus  cue  and  response  tool  were  incongruent  (i.e.,  static,  motion  onset  cue 
paired  with  dynamic  response  tool  or  the  dynamic,  motion  direction  cue  paired  with  the 
static  response  tool)  and  only  32.2  ms  faster  when  the  stimulus  cue  and  response  tool 
were  congruent  (i.e.,  both  static  or  both  dynamic);  see  Figure  2. 

Spatial  Compatibility  and  Handedness 

Overall,  spatially  compatible  responses  were  significantly  faster  than  incompatible 
responses  [374.7  420.0  ms;  F(l,12)  =  243.7,  p  =  .0001].  Compatibility  interacted 

with  handedness  [F(l,12)  =  7.81,  p  =  .016];  see  Figure  3.  This  interaction  means  that 
left-handed  subjects  showed  a  larger  compatibility  effect  (incompatible  -  compatible  RT 
=  53.4  ms)  than  right-handed  subjects  (incompatible  -  compatible  RT  =  37.3  ms). 
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I  Cue  -Tool  Compatibility  x  Handedness 


tool 

dynamic 

static 


static 

dynamic 


Figure  2:  Mean  response  times  of  matched  groups  of  left-handed  and  right-handed 
subjects  for  compatible  and  incompatible  cue-tool  conditions.  Both  groups  of  subjects 
show  shorter  RTs  for  the  compatible  condition,  the  decrease  in  RT,  however,  is  much 
more  pronounced  for  left-handers. 


I  Spatial  Compatibility  x  Handedness  | 


Figure  3:  Mean  response  times  of  matched  groups  of  left-handed  and  right-handed 
subjects  for  compatible  or  incompatible  assignment  of  stimuli  to  responses.  As  in  Fig,  2 
left-handers  show  a  larger  drop  in  RT  with  compatible  conditions  than  right-handers. 
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Discussion 

The  present  study  shows  that  performance  depends  on  the  compatibility  between 
stimulus  cue  and  response  tool  and  on  the  spatial  compatibility  of  assignment  between 
the  locations/directions  of  stimuli  and  responses^\  Moreover,  these  two  compatibility 
effects  interact  with  handedness. 

Compatibility 

We  have  restricted  the  application  of  the  compatibility  principle  to  rather  simple 
situations  to  maintain  experimental  clarity.  While  the  effect  of  compatible  or 
incompatible  assignment  of  spatial  dimensions  location  or  directions)  of  stimuli  to 
responses  has  been  studied  before  (see  Proctor  and  Reeve,  1990,  for  a  general  review; 
and  Ehrenstein,  1994,  for  a  distinction  between  location  and  direction  compatibility)  the 
cue-tool  compatibility  is  a  new  effect  which  deserves  further  investigation.  This  new 
compatibility  effect  adds  to  various  further  applications  of  the  compatibility  principle 
which  comprise  problems  such  as  cursor  control  on  a  computer  screen  (Gude,  1993), 
control  of  manual  tracking  (Cavonius  et  al.,  1996),  spatial  compatibility  in  various 
orientations,  including  orthogonal  stimulus-response  relationships  (Ehrenstein  et  al,, 
1989;  1996),  or  proximity  compatibility  (Wickens  &  Carswell,  1995).  Despite  the 
reasonable  assumption  that  all  of  these  compatibility  effects  interact  with  (or  are  at  least 
modified  by)  the  respective  eye-dominance  relationships  of  the  operator,  this  issue  has 
scarcely  been  addressed  to  date.  Conversely,  the  vast  literature  on  handedness  and  on 
laterality  of  sensory  or  motor  functions  in  general  (e.g.,  Schmauder  &  Solf,  1992;  Peters, 
1995;  Bourassa  et  al.,  1996)  has  not  yet  tapped  the  issue  of  compatibility  and  its 
possible  implications  for  the  performance  of  lateralized  functions. 

Eye-Dominance 

Although  we  obtained  no  significant  effects  of  eye-dominance  there  was,  on  average,  a 
tendency  for  right-eyed  persons  to  show  a  response-time  advantage  over  left-eyed 
persons.  One  reason  for  statistical  insignificance  is  the  obviously  small  sample  of 
subjects  examined.  In  fact  in  a  further  study  (Amold-Schulz-Gahmen  et  al.,  1997)  ocular 
dominance  was  found  to  affect  suprathreshold  motion  sensitivity.  Another  reason  for  the 
absence  of  an  eye-dominance  effect  in  the  present  study  may  be  due  to  the  fact  that 
observation  was  always  binocular:  differences  between  eyes  can  be  expected  to  show  up 
more  clearly  in  monocular  tasks,  i.e.,  if  the  performance  is  tested  for  both  eyes 
separately.  Interestingly,  preliminaiy  results  suggest  that  the  difference  between 
monocular  and  binocular  performance  is  much  more  pronounced  in  right-eyed  then  left¬ 
eyed  subjects  (Amold-Schulz-Gahmen  et  al.,  1997). 


1)  The  cue-tool  correspondence  can  be  considered  as  a  special  case  of  compatibility,  that  of  set-level 
compatibility:  for  a  natural  correspondence  between  stimulus  sets  and  response  sets,  as  opposed  to 
element-level  compatibility  between  arbitrarily  defined  pairings  of  indvidual  stimuli  and  responses  (Fitts 
and  Seeger,  1953;  see  also  Proctor  and  Dutta,  1995).  In  the  present  case,  set-level  compatibility  occurs 
between  (static/dynamic)  stimulus  cues  and  (static/dynamic)  response  tools.  Thus,  one  may  distinguish 
here  a  cue-tool  compatibility  from  a  spatial  compatibility  which  occurs  between  spatial  dimensions  or 
directions  of  stimuli  and  responses  (left/right,  leftward/rightward). 
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Handedness 

Left-handers  were  slower  than  right-handers,  especially  with  incongruent  cue-tool 
relationship  and  incompatible  assignment  between  stimuli  and  responses.  This  means 
that  their  disadvantage  was  less  if  conditions  for  perception-action  compatibility  were 
optimal.  In  other  words,  an  ergonomic  design  that  improves  the  workplace  design  on  the 
basis  of  the  compatibility  principle  alone  would  already  improve  the  situation  for  both 
right-handed  and  left-handed  persons.  Certainly,  for  left-handed  subjects  the  workplace 
design  can  be  further  improved  if  it  specifically  addresses  the  issue  of  left-handedness 
(see  Schmauder  &  Solf,  1992,  for  a  review). 

Outlook 

The  present  study  provides  exploratory  rather  than  definite  evidence.  As  mentioned 
before,  further  research  should  be  conducted  to  determine  whether  eye-dominance  effects 
influence  performance  if,  instead  of  binocular  vision,  monocular  vision  were  required,  so 
that  the  dominant  eye  and  the  non-dominant  eye  were  employed  selectively.  The  same 
holds  for  the  effect  of  handedness.  Our  study  suggests  that  the  RT  advantage  for  the 
dominant  hand  was  stronger  when  the  task  engaged  both  hands  (key  response)  than  when 
only  one  hand  was  used  (joystick  response).  It  needs  to  be  shown  in  separate 
experiments,  which  compare  bimanual  vwth  unimanual  tasks  using  the  same  rather  than 
different  response  tools,  to  what  extent  hand-dominance  effects  depend  on  this  factor.  In 
addition,  the  aspect  of  genuine  bimanual  coordination  (e.g.,  Heuer,  1990;  Heuer  et  al., 
1997;  Spijkers  &  Heuer,  1995)  and  hand-dominance  should  be  specifically  addressed  by 
future  research.  Thus,  the  present  study  just  opens  the  door  to  a  rich  field  of  research 
questions  that  links  issues  of  compatibility  to  that  of  dominance  of  eye  and  hand  with  the 
aim  to  optimize  the  design  of  an  individual’s  workplace.  In  addition  this  future  research 
would  help  greatly  to  become  aware  of  the  fact  of  dominance,  especially  of  eye- 
dominance.  Many  of  our  subjects  were  puzzled  in  that  they  did  not  know  what  was  their 
dominant  eye  or  even  had  never  heard  that  something  like  eye-dominance  exists.  Thus,  a 
routine  screening  of  eyedness  and  handedness  by  a  short  questionnaire  (e.g.,  Coren  1993; 
German  adaptation  by  Ehrenstein  &  Amold-Schulz-Gahmen,  1997)  is  recommended  for 
each  employee. 
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